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FOREWORD 


This  report  presents  the  findings  of  an  investigation  performed 
to  examine  United  States  Air  Force  contracting  and  acquisition  techniques 
utilized  to  assure  the  quality  and  reliability  of  weapon  systems.  It  was 
performed  by  the  International  Technology  Corporation  (INTEC)  at  the 
request  of  the  United  States  Air  Force  Business  Research  Management  Center. 
The  principal  objective  of  the  study  was  to  examine  the  general  motivational 
aspects  of  the  USAF  contracting  techniques  with  particular  attention  to 
quality  assurance  and  reliability  policies,  management  techniques  and 
practices. 


The  study  methodology  consisted  of  three  phases:  an  examination 
of  current  qual  ity/rel labil ity  policy;  interviews  with  USAF  procurement  and 
contract  administration  officials  and  industry  officials;  and  a  wide-gauged 
industry  questionnaire.  The  questionnaire  was  designed  to  obtain  an 
industry-wide  viewpoint  of  quality/reliability  policy.  Existing  DoD  policy 
was  reviewed  and  a  comprehensive  literature  review  was  undertaken  to  identify 
other  research  efforts  in  the  product  quality  motivational  arena.  Following 
completion  of  these  tasks,  the  results  were  integrated  to  determine  what, 
if  any,  recommendations  should  be  made. 

The  investigation  resulted  in  the  following  primary  recommendations: 

o  Establish  a  central  contractor  product  history 
f  i  1  e , 

o  Adopt  the  proposed  Cost  Plus  Award  Fee  (quality) 
methodology,  and 

o  Adopt  certain  recommended  DAR  revisions  relating 
to  motivational  concepts  for  contracting. 

Implementing  documentation  for  each  of  the  above  recommendations  is  included 
in  this  report.  In  addition,  a  USAF  Acquisition  Managers  Quality  Guide  was 
prepared  and  is  included  as  a  separate  document. 

We  are  indebted  to  the  project  manager,  Major  Lyle  Lockwood,  for 
his  continuing  technical  consultation  and  assistance.  In  addition,  we  are 
sincerely  appreciative  of  the  assistance  of  all  of  the  other  Air  Force  and 
contractor  personnel,  who  devoted  their  time  and  expertise  during  the  period 
of  this  investigation. 

The  findings,  conclusions  and  recommendations  of  this  report  should 
be  viewed  as  a  reflection  of  opinions  of  individuals  surveyed  and  not  as 
current  or  projected  policies  of  the  organizations  participating  in  the  survey. 
In  addition,  the  comments,  conclusions  and  recommendations  of  this  report 
represent  the  professional  views  of  the  contractor  who  conducted  the  survey 
and  do  not  necessarily  represent  those  of  the  BRMC  Office. 
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I.  GENERAL  CHARACTERISTICS  OF  THE  DEFENSE 


MARKET  AND  PROCUREMENT  METHODOLOGY 
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i.  general  characteristics  of  the  defense 

MARKET  AND  PROCUREMENT  METHODOLOGY 


A.  GENERAL  • 

An  evaluation  of  industrial  motives  which  cause  the  firm  to 
participate  successfully  in  the  Defense  market  must  include  an  examination 
of  the  market  and  its  peculiar  characteristics.  The  FY  1980  Department  of 
Defense  budget  contains  projected  expenditures  of  $35  billion  for  procure¬ 
ment.  Another  $13  billion  was  projected  for  research  development  tests  and 
evaluation  making  a  total  of  $48  billion.  The  magnitude  of  this  market  is 
sufficient  attraction  in  itself  to  cause  contractors  to  wish  to  participate 
in  it.  There  are  other  factors  as  well.  Some  examples  include:  high 
technology  transferable  to  commercial  products,  relatively  low  risk  market, 
diversity,  patriotism,  the  only  market  for  the  firm's  product  line  and 
prestige.  A  collateral  benefit  is  the  development  and  training  of  manage¬ 
ment  and  engineering  personnel.  These  factors,  and  others  combined  with 
market  size  continually  attract  firms  to  participate  in  the  Defense  market. 

B.  DEFENSE  MARKET  CHARACTERISTICS 

The  "classic  market"  is  characterized  by  producers  with  products 
that  are  ready  and  deliverable  and  buyers  who  decide,  based  on  the  character¬ 
istics  of  the  product  and  their  needs,  whether  to  buy  or  not.  The  buyer/ 
seller  confrontation  results  in  a  quick  and  firm  decision  on  price,  and 
failure  of  the  product  once  procured  will  have  an  immediate  and  negative 
impact  on  the  survivability  of  the  producer. 

The  Defense  market,  in  general,  and  certainly  in  the  case  of 
systems,  differs  partly  from  the  classic  market  approach  due  to  the 
frequent  utilization  of  an  administrative  determination  of  price  based 
on  costs  incurred  by  the  contractor,  i.e.,  cost  reimbursement  contracts. 

It  differs  also  in  that  the  product  may  only  be  definable  in  general  terms, 
thus  creating  technical  uncertainties  which  may  lead  to  major  difficulties. 
Also,  the  producers  responsibilities  tend  to  be  limited  by  these  same 
uncertainties.  The  user  of  the  product  is  not  the  buyer  and  the  diversity 
of  the  use  environment  makes  the  application  of  warranty  type  of  penalties 
difficult  to  manage  equitably.  The  Defense  market  is  also  different  in  that 
the  products  are  often  unique  and  not  directly  saleable  in  the  general  market. 
In  some  cases  though,  technological  advances  may  be  made  which  are  transfer¬ 
able  to  the  private  sector  thus  providing  a  competitive  advantage  in  that 
market  and  a  possibly  important  motivation  to  participate  in  the  Defense 
market. 


All  of  these  and  other  unique  characteristics  have  resulted  in  a 
variation  in  the  classic  market  model.  This  unique  defense  market  has  led 
to  the  emergence  of  a  group  of  firms  who  have  adapted  themselves  to  defense 
market  characteri sties  and  have  specialized  in  their  activities  so  that  they 
can  operate  successfully.  In  the  defense  environment  many  of  the  successful 
government  contracting  firms  are  elements  of  major  firms  who  are  also 
successful  in  the  traditional  commercial  market  place. 
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The  military/government  procurement  process  is  subject  to 
several  stresses.  First  is  the  inherent  requirement  for  purchasing 
officials  to  be  responsible  in  procurement  actions  and  to  be  responsive 
to  the  public  trust  which  falls  to  public  servants  who  expend  government 
funds.  Second  .is  an  accompanying  obligation  to  operate  in  a  framework  of 
absolute  fairness  as  opposed  to  the  competitive  situation.  Thirdly,  there 
is  a  continuing  broad  and  high  level  review  of,  and  variation  in,  procure¬ 
ment  policy  as  a  result  of  changing  political  emphasis,  e.g.,  total  package 
procurement,  incentive'  contracting,  and  "fly  before  buy."  There  are  also 
reviews  of  particular  procurements  which  may  be  required  in  response  to 
special  requests  from  other  government  officials. 

A  continuing  emphasis  of  the  Defense  Acquisition  Regulation 
(DAR),  is  embodied  in  the  clearly  stated  premise  that  profit-is  the  prime 
motivator,  accompanied  by  unstated  but  well  established  punitive  motivational 
concepts  as  evidenced  by  such  clauses  as  Liquidated  Damages.  Positive  and 
punitive  methods  for  motivating  contractors  have  historically  been  major 
alternate  motivational  elements  of  the  free  market  system.  They  have  tra¬ 
ditionally  been  successful  as  motivating  approaches  to  assure  the  delivery 
of  conforming  material,  delivery  on  time,  and  at  the  agreed  upon  price. 

The  free  market  concept  as  it  applies  to  system  procurement, 
has  been  discounted  by  other  researchers,  as  stated  by  Frederic  M.  Scherer 
..."attributes  of  weapons  acquisition  preclude  reliance  on  anything  like  a 
conventional  market  system  for  the  procurement  of  advanced  weapons,  evoking 
instead  what  is  best  described  as  a  non-market,  quasi -admi ni strati ve , 
buyer-seller  relationship.  In  this  non-market  environment,  the  autocratic 
guides  and  restraints  provided  by  the  market's  'invisible  hand'  are  absent. 

To  replace  them  the  government  must  deliberately  structure  its  relations 
with  contractors  in  such  a  way  as  to  assure  successful  weapons  program 
execution."1 

C,  AEROSPACE  MARKET  CHARACTERISTICS 

The  American  airfleet  is  comprised  of  three  major  segments.  The 
first  is  the  commercial  air  carriers  and  is  about  2,500  aircraft,  the  second 
is  the  military  airfleet  of  approximately  20,000.  The  largest  is  the  general 
aviation  group  of  private  aircraft  of  approximately  161,500,  divided  between 
recreational  aircraft  and  private  aircraft  operated  by  business  or  other  types 
of  users.  The  total  market  then  represented  by  the  American  general  aviation 
airfleet  comprises  about  184, 0002  aircraft.  This  does  not  include  the 
foreign  market. 

The  Federal  Aviation  Administration  (FAA)  is  the  principal  arm  of 
government  which  controls  the  design  and  manufacture  of  aircraft  and  associated 


1  Scherer,  Frederic  M.,  Weapons  System  Acquisition  Process:  Economic 
Incentives ,  Boston:  Harvard  University  Press,  1964 

2  Graham,  L.  J.,  Retterer,  B.  L.,  Airline  Procurement  Techniques,  ARINC 
Research  Corporation,  1976 
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subsystems.  The  FAA  authority  derives  from  the  Federal  Aviation  Act  of 
1958  which  established  the  authority  of  the  agency  for  the  safe  and 
effective  operation  of  the  national  aviation  system.  Several  missions 
are  assigned  the  agency  which  impacts  those  manufacturers  who  are  in  the 
aviation  market' place .  Principal  FAA  quality  assurance  missions  are: 

o  Type  certification  of  aircraft  which  confirms 
that. aircraft  designed  for  civil  use  meet  FAA 
standards  of  construction  and  performance. 

o  Production  certificates  which  attest  to  the 
manufacturers1  ability  to  duplicate  the  design 
under  an  FAA  approved  quality  program. 

o  Air  worthiness  certification  which  is  required 
of  every  airplane  that  is  produced. 

These  responsibilities  of  the  FAA  do  not  apply  in  total,  of  course,  to  the 
military  aircraft  market.  However,  they  do  have  a  definite  impact  on  procurers 
of  aeronautical  equipment  purchased  by  the  Air  Force  Systems  Command  (AFSC) 
and  should  be  considered  as  an  asset  to  military  purchasers  of  these  equipment. 

D.  AVIONICS 


The  avionics  market  supplies  both  the  military  and  private 
sectors.  The  private  sector  is  divided  into  two  subgroups  (commercial 
and  private  aircraft).  Private  aircraft  is  further  subdivided  into 
usiness,  e.g.,  corporate  aircraft  and  recreational  aircraft. 


Avionics  may  be  described  as  a  high  technology,  labor  intensive 
product.  Avionics  equipment  technology  has  been  revolutionized  since  World 
War  II.  The  advent  of  solid-state  amplifiers  replacing  vacuum  tubes  is  only 
the  top  of  the  technological  iceberg  that  the  avionics  industry  has  encountered. 
Foreign  products  have  penetrated  the  market  providing  American  firms  a  strong 
incentive  to  excel  technologically  and  to  drive  prices  down. 


Military  procurements  frequently  attempt  to  expand  the  known  perfor¬ 
mance  limits  possible  with  existing  technology  and  the  military  market  often 
buys  in  large  quantities  as  well.  This  is  in  contrast  to  the  airline  market 
ere  most  carriers  purchase  in  small  quantities  to  meet  annual  demands.  The 
private  sector  is  also  more  willing  to  settle  performance  wise  for  current 
state-of-the-art,  or  perhaps  slight  advances.  The  civilian  market  is  also 
highly  predictable  and  the  procurement  process  is  more  straightforward.  The 
market  for  civilian  avionics  is  more  likely  to  be  concentrated  on  form  fit 
and  function,  i.e.,  interchangeability  and  reliability.  It  also  carries  with 
it  considerable  support  services  such  as  warranty  work,  spares  provisioning, 
training,  and  on-site  technical  assistance. 


A  major  difference  is  the  tendency  for  the  air  carrier  to  stay  with 
a  single  supplier  and  to  cooperate  closely  with  that  supplier  on  finding 
solutions  to  significant  technical  difficulties  that  may  arise.  In  short, 
the  civilian  market  is  based  on  assuring  the  availability  of  the  function 
the  equipment  provides  rather  than,  as  in  the  military  case,  simply  providing 
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a  product.  The  civilian  market  is  far  more  interested  in  a  product  that 
will  continue  to  operate  whereas  the  military  is  often  driven  by  "state- 
of-the-art"  advancement  requirements  and  the  necessity  to  field  systems 
which  avoid  early  obsolescence.  The  military  environment  is  one  of 
technical  competition  wherein  the  objective  often  is  to  leap-frog  the 
technology  of  the  newest  existing  system. 

The  USAF  purchases  equipment  for  various  reasons  throughout  the 
life  cycle  including  the  fol’owing:  (1)  for  i ncorporation  into  systems  in 
the  development/production  phase,  (2)  for  replacements  for  fielded  systems 
beyond  the  development  phase,  and  (3)  for  a  multiplicity  of  applications 
in  other  than  weapon  systems.  The  equipment  is  bought  as  well  by  other  DoD 
organizations  and  by  other  government  and  non-government  organizations.  The 
posture  of  the  avionics  producer  relative  to  its  customer  regulatory 
environment  may  be  seen  in  Figure  1-1 . 

Figure  1-1 

Relationship  of  Avionics  Manufacturer 
to  Customer/ Regulatory  Environment 


The  customers  shown  in  Figure  1-1  seek  to  have  some  control  over 
the  development  production  processes  that  will  provide  some  assurances  that 
the  equipment  will  function  within  predetermined  operational  parameters,  e.g., 
frequency,  power. and  sensitivity.  This  is  frequently  furthered  by  specifying 
a  specific  technical  discipline  and  manufacturing  procedures/processes  that 
will  assist  in  assuring  that  the  equipment  has  a  high  probability  of  performing 
the  function  successfully.  The  avionics  manufacturer ' s  customers  generally 
force  an  acceptance  of  agreed  upon  procedures  as  a  condition  of  the  procurement 
being  considered. 
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Before  a  manufacturer  can  produce  an  avionics  product  to  be  used 
on  civil  aircraft,  authority  must  be  granted  by  the  FAA.  The  authorization 
is  issued  based  on  conformance  with  FAA  Regulation,  Volume  2,  Paragraph  37 
and  the  Applicable  Technical  Standard  Order  (TSO).  The  TSO  contains 
minimum  performance  and  quality  control  standards  for  equipment  to  be  used 
on  civil  aircraft.  Once  the  TSO  has  been  granted,  the  manufacturer  must 
produce  the  equipment  in  accordance  with  his  application,  conduct  all  required 
tests  and  inspections  and  establish  and  maintain  a  quality  assurance  system 
adequate  to  assure  the  equipment  meets  requirements.  At  any  time,  the 
manufacturer  is  subject  to  inspections  of  his  quality  control  system,  his 
manufacturing  facilities,  and  his  records  by  the  FAA. 

In  addition  to  the  FAA  regulatory  enforcement  activities  described 
above,  the  FAA  has  its  own  quality  assurance  program  for  material/equioment 
purchased  for  its  own  use.  These  programs  are  described  in  FAA  Standards 
013,  016  and  018.  Enforcement  instructions  for  quality  assurance  personnel 
are  described  in  FAA  Orders  4630.8  and  4453. 2A.  The  FAA  quality  assurance 
programs  are  largely  based  on  the  military  programs,  e.g.,  MIL-Q-9858. 

E.  BUYING  OFFICES 

The  procurement  of  avionics  for  Air  Force  purposes  is  frequently 
managed  at  the  plant  level  by  the  Defense  Logistics  Agency.  The  Defense 
Logistics  Agency  enforces  whatever  requirement  the  Purchasing  organization 
includes  in  the  contract.  It  is  done  in  accordance  with  the  DLA  in-plant 
manual,  DLAM  8200.1.  The  procurements  in  any  given  avionics  manufacturing 
facility  may  come  from  several  sources  outside  the  AFSC  including  AFLC  and 
several  AFSC  division  offices  as  well.  These  relationships  are  depicted 
in  Figure  1-2. 

The  intensity  of  management  control  from  the  various  purchasing 
offices  covers  a  wide  spectrum  of  interests  and  controls.  A  project  office 
facing  a  system  schedule  will  have  a  different  perspective  than  an  Air 
Logistics  Center  (ALC)  supporting  a  deployed  system.  The  urgency  of  need  is 
manifested  in  varying  degrees  of  pressure  exerted  on  the  manufacturer. 

(However,  it  is  unlikely  that  government  purchasing  offices  will  exert  the 
pressure  that  the  commercial  airline  will  that  has  negotiated  a  clear  warranty 
and  has  operational  aircraft  grounded  due  to  a  failure.)  In  commercial 
cases,  warranty  claims  are  expeditiously  processed  on  a  daily  basis  and 
supplies,  repair  costs,  and  turn-around  time  carefully  monitored.  This  can 
be  contrasted  to  military  warranties  and  the  protracted  negotiations  that 
surround  them. 

The  procurement  and  contract  administration  process  in  the  private 
sector  are  far  less  complicated  and  protracted  than  in  the  military  sector. 

While  there  is  considerable  upstream  activities,  particularly  in  developing 
new  products,  the  commercial  purchasing  group  controls  the  process  subsequent 
to  the  development  of  the  preliminary  specification.  Assistance  or  involvement 
ot  customers  in  the  manufacturing  environment  concentrates  primarily  on  problems 
of  performance  or  application  as  opposed  to  manufacturing  practices.  Quality 
assurance  personnel  are  mainly  interested  in  the  reliability  and  maintainability 
of  the  product.  Quality  assurance  activity  in  the  pre-contractual  period  is 
most  predominant  in  the  proposal  evaluation  and  performance  evaluation  stages. 
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Figure  1-2 


Relationship  of  Avionics  Manufacturer  to  USAF  and 
Other  Customer/Regulatory  Groups 


After  product  delivery,  quality  assurance  monitors  performance  of  product 
against  the  warranty.  There  is  limited  quality  assurance  involvement 
i n  the  manufacturing  process.  In  the  event  of  performance  levels  not 
being  achieved,  quality  assurance  analysis  may  lead  to  modifications  or 
other  recommendations.  There  is  notable  contrast  here  in  emphasis  as 
compared  with  government  purchasing  offices,  including  the  FAA,  but 
particularly  with  DoD/USAc  offices  who  concentrate  on  what  might  be 
called  the  industrial/manufacturing  practices  aspects.  Figure  1-3  below 
illustrates  that  commericial  airline  quality  emphasis  is  relatively 
great  in  the  early  phases  of  the  life  cycle  with  considerable  involvement 
after  deployment.  The  commercial  carriers  reliance  on  FAA  controls 
permit  quality  assurance  personnel  to  devote  more  emphasis  in  the  early 
phase  and  the  later  emphasis  reflects  their  involvement  in  the  warranty 
process.  This  phasing  of  effort  is  in  obvious  constrast  to  the  USAF 
concentration  of  activity  in  the  production  phase. 

Figure  1-3 

USAF  Versus  Commercial  Quality  Assurance  Level  of  Effort 


Commercial  Carrier  QA  •  •  •  • 
USAF  QA  — - 


F,  IDENTIFICATION  OF  PRODUCT  CATEGORIES 

The  selection  of  product  categories  is  important  to  this  study 
in  that  it  provides  a  baseline,  or  focus  about  which  to  build  an  analysis 
of  current  field  performance  and  contractor  motivational  considerations 
in  relation  to  contracting  methodology. 
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The  elements  of  AISC  that  were  initally  under  study  included 
the  Electronic  Systems  Division,  The  Aeronautical  Systems  Division,  The 
Armament  Division,  The  Space  Division  and  The  Ballistic  Missile  Office. 

A  review  of  commodities  purchased  by  those  commands  shows  that  in  excess 
of  100  classes  of  equipment  are  being  procured. 


Review  of  the  classes  showed  that  they  could  successfully 
and  logically  be  organized  into  a  smaller  number  of  subgroups.  These 
subgroups  are  electronic,  air  frame,  propulsion,  armament  and  mechanical. 
The  mechanical  subgroup  was  dropped  initially  since  it  became  apparent 
that  many  mechanical  devices  were  associated  with  air  frame  or  (vehicular 
body  structure)  or  electronics  or  propulsion  and  are  developed  and 
purchased  under  the  central  direction  and  control  of  the  air  frame  con¬ 
tractor.  Armament  systems  also  contain  a  large  element  of  electronics 
and  although  mechanical  components  are  included,  they  are  considered  to 
be  relatively  minor  in  terms  of  unique  contracting  procedures.  The  air 
frame  group  seemingly  offered  what  appeared  to  be  an  opportunity  for 
measuring  the  impact  of  contractual  quality  motivators,  particularly  with 
respect  to  subcontractors.  The  availability  of  ready  data,  however, 
reduced  the  priority  of  this  subgroup  below  the  remaining  two  subgroups, 
electronics  and  propulsion. 

Electronic  devices  permeate  practically  all  system  components, 
however,  it  is  most  typically  represented  by  avionics  and  it  is  here  that 
much  innovative  development  of  testing  and  procurement  approaches  has 
occurred.  While  the  propulsion  system  has  less  electronics  than  any  other 
subsystem,  even  it  has  some  dependence  on  electronic  controls.  This  sub¬ 
system  stands  out  as  a  mechanical  device  about  which  major  procurement 
and  testing  techniques  have  been  developed.  For  these  reasons,  avionics 
and  engines  were  believed  to  offer  the  best  opportunities  as  the  product 
categories  used  for  studying  procurement  methods,  and  contractor  motivators 
related  to  quality  and  reliability.  Accordingly,  they  are  used  as  examples 
throughout  the  balance  of  this  section. 

G.  REPRESENTATIVE  FIELD  QUALITY  AND  RELIABILITY  LEVELS 

In  an  ideal  world,  one  would  satisfy  the  objectives  of  this 
study  by:  identifying  the  contracting  methodologies  used  for  a  variety 
of  programs;  analyzing  the  market  conditions  under  which  the  contracts 
were  negotiated  and  administered;  evaluating  the  resultant  quality  levels 
achieved;  and  selecting  those  existing  or  required  new  methodologies 
which  will  provide  the  maximum  motivation  to  the  .contractor  to  produce 
high  quality  equipments/systems.  The  realities  of  the  actual  procurement 
situation,  however,  are  far  from  being  clear  cut.  Although  the  written 
contracting  methodologies  for  a  group  of  contracts  can  be  researched 
and  documented,  only  the  customer  and  contractor  program  managers  on 
each  contract  can  begin  to  construct  how  effectively  the  written  word 
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was  enforced,  bargained  away,  circumvented,  treated  adequately  or 
superficially,  or  even  gambled  upon  in  the  face  of  contrary  technical 
data  that  strongly  suggested  that  inadequate  performance  is  to  be 
expected.  The  market  conditions  may  be  analyzed  but  the  response 
of  the  customer  and  each  contractor  to  the  stimuli  based  upon  these 
conditions  is  unique  and  will  change  differently  with  respect  to  time. 
Finally,  the  development  of  a  non-amb.iguous  measure  of  resultant  field 
quality  which  is  relatable  to  the  success  or  failure  of  a  contracting 
methodology  is  far  from  straight  forward. 

In  the  realities  of  system  performance,  the  only  currently 
available  practical  measure  of  field  quality  and  reliability  levels  is  mean 
time  between  failure.  On  this  premise,  steps  were  taken  to  receive  failure 
data  from  AFLC  as  reported  in  the  DO  56  Product  Performance  System  Data 
products.  It  was  further  intended  that  data  would  relate  not  only  to 
engines  and  selected  avionics  subsystems  or  components  but  would  relate 
to  different  types  of  aircraft  as  well.  The  results  and  findings  of 
this  investigation  were  as  follows: 

1  .  Avionics 

In  the  case  of  electronic  equipment,  the  data  was  examined 
for  the  following  aircraft:  A7,  B52,  C5,  Cl  30 ,  F4 ,  F5,  F15,  F16, 

Mil,  and  J39.  Further  examination  of  the  data  showed  such  dispersions 
of  reliability  figures  between  aircraft  that  analysis  based  on  the  DO  56 
product  would  need  to  be  supplemented  by  extensive  visits  and  communications 
vith  both  producers  and  users.  These  dispersions  are  due  to  numerous 
factors  including  but  not  limited  to  the  following: 

o  The  equipment  interfaces  in  different  aircraft 
cause  variances  in  failure  rates 

o  Mission  differences  result  in  varying 
environmental  stress  applications 

o  Varying  competence  levels  of  support  activities 

o  Applications  of  equipment  in  environments  which 

were  unknown  by  the  designers  and,  therefore  result  in 
different  failure  levels 

o  differences  in  failure  reporting  and  feedback 
effectiveness . 

In  recognition  of  the  interfaces  between  policy  and  product  quality 
(rel iabil ity)  and  the  continuing  high  level  interest  in  reliability  and 
the  risks  inherent  in  utilizing  66-1  data  as  the  only  data  source,  it  was 
determined  to  utilize  data  made  available  from  the  Rome  Air  Development 
Center. 3 


3  Rome  Air  Development  Center,  Operational  Influences  on  Reliability,  RADC 
TR  76-366,  December  1  976  '  J~ 
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The  data  in  this  report  was  the  result  of  an  investigation  which 
extended  over  almost  two  years  utilizing  AFM  66-1  MDC  system  data, 

AFM  65-110  data,  K051  system  data  products  and  the  DO  56  Product 
Performance  System  Data  products-.  Logistics  performance  factors  were 
obtained  using  the  method  described  in  AFLCM  800-3. 

The  data  base  used  for  this  study  represented  sixteen  different 
items  of  USAF  avionics  equipment  in  operational  use  on  ten  different  USAF 
aircraft  weapons  systems.  Since  many  of  these  equipments  are  used  on  two 
or  more  different  weapons  systems,  a  total  of  thirty  different  applications 
(combinations  of  equipment  and  weapon  system)  were  included  in  the  study. 

A  matrix  indicating  the  equipment  items  and  the  applications  on 
each  of  the  systems  is  presented  in  Table  1-1 

Table  1-1 

Equipment/Weapon  System  Combinations 


NUMBER 

FUNCTION 

EQUIPMENT/WEAPON  A 

NO  SYSTEM  COMBINATIONS 

A 

B 

c 

D 

F 

H 

J 

K 

N 

p 

SAC 

TAC 

MAC 

ATC 

SAC 

ADC 

SAC 

TAC 

TAC 

TAC 

1 

Comm 

0 

0 

0 

0 

0 

0 

2 

Comm 

0 

3 

Nav 

0 

0 

4 

Nav 

0 

0 

5 

Radar 

0 

6 

Radar 

0 

0 

0 

7 

Radar 

0 

8 

Radar 

0 

9 

Radar 

0 

10 

Nav/Radar 

0 

11 

Nav/Radar 

0 

0 

0 

12 

Nav/Radar 

0 

0 

13 

Nav/Radar 

0 

0 

14 

Instr 

0 

15 

Nav/Radar 

0 

0 

16 

Instr 

0 

TOTAL 

EQUIPMENTS 

5 

3 

4 

1 

2 

2 

2 

3 

6 

2 
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Table  1-2  presents  a  summary  of  field  MTBF  values  for  the  16 
avionics  equipments.  For  reference  purposes,  the  required  MTBF  values 
are  also  shown  along  with  various  ratios  of  predicted,  demonstrated  and 
field  MTBFs  as  well  as  the  MTBF  actually  experienced  in  the  deployed 
equipment. 


Table  1-2 

Avionics  Equipment  MTBF 


EQUIPMENT 

NO. 

REQUIRED 

MTBF 

PREDICTED 

REQUIRED 

DEMONSTRATED 

FIELD 

FIELD  MTBF 

REQUIRED 

REQUIRED 

1 

450 

2.43 

1  .23 

0.44 

198 

2 

1500 

0.94 

1  .00 

0.32 

480 

3 

1000 

2.90 

0.67 

0.14 

140 

4 

700 

3.71 

1  .25 

0.51 

357 

5 

30 

1  .10 

1  .25 

0.63 

19 

6 

193 

1  .48 

1  .06 

0.64 

124 

7 

83 

2.25 

1  .59 

0.32 

27 

v  8 

75 

3.44 

1  .21 

1  .54 

115 

9 

475 

3.00 

1  .72 

0.11 

52 

10 

600 

1  .05 

0.96 

0.21 

126 

11 

770 

4.27 

1  .30 

0.46 

354 

12 

300 

2.25 

1  .35 

0.37 

111 

13 

140 

1  .01 

1  .32 

1  .53 

214 

14 

380 

3.81 

1  .42 

0.60 

228 

15 

900 

0.98 

0.99 

0.62 

558 

16 

1040 

1  .28 

0.59 

0.55 

572 

2 .  Propul  si  on 

MTBF  values  for  the  jet  engine  category  were  developed  from  the 
six  month  period  ending  in  March  1980,  from  the  DO  56  Product  Performance 
System  Data  Products,  ihis  data  was  drawn  directly  from  the  report  without 
the  benefit  of  the  extensive  background  data  available  for  the  avionics 
analysis.  It  is  expected  that  many,  if  not  all,  of  the  same  factors  affecting 
the  MTBF  relationships  of  the  avionics  category  also  have  some  impact  upon  the 
engine  category,  although  the  impacts  of  these  factors  may  be  different  and 
there  may  also  be  additional  factors  as  well.  The  engine  MTBF  values  are 
shown  by  aircraft  type  in  Table  1-3  below. 
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Table  1-3 


Field  MTBF  Values  by  Aircraft  Type 


Ai rcraft 

Engi nes 

MTBF 

Operating  Hours 

T-38 

•  J-85 

155 

140,111 

C-5A 

TF  39 

2786 

25,075 

F-4 

J-79 

34 

3,739 

F-4E 

J-79 

8321 

66,566 

F-5 

J-85 

153 

1  ,375 

F-15 

F-100 

17 

46,188 

F-111A 

TF-30 

495 

8,413 

F-111F 

TF-30 

360 

9,729 

F-111D 

TF-30 

275 

_  9,340 

F-105 

J— 7  5 

8158 

16,315 

F-100 

J-57 

85 

85 

F-16B 

F-100 

2942 

2,942 

B-52G 

J-57 

177 

30,805 

Reliability  is  the  established  method  for  specifying  and 
measuring  time  and  performance  relationships.  It  is  also  recognized  that 
unanticipated  operating  failures  account  for  a  large  proportion  of  support 
costs.  In  RADC  TR76-366,  it  was  observed  that  maintenance  actions 
attributable  to  avionics  equipment  account  for  only  61  percent  of  the 
life  cycle  costs  while  maintenance  actions  on  interface  and  related 
hardware  accounted  for  the  other  39  percent.  This  demonstrates  that 
39  percent  of  equipment  support  cost  could  not  be  eliminated  even  if 
there  were  no  avionics  failures.  This  is  obviously  an  area  where 
additional  quality  (reliability)  management  attention  or  policy  may 
be  advisable. 


11 


II.  quality  ano  reliability  policies  and  practices 


II.  QUALITY  AND  RELIABILITY  POLICIES  AND  PRACTICES 


♦ 


A.  GENERAL 

This  part  of  the  report  includes  an  analysis  of  the  quality  and 
reliability  requirements  imposed  both  on  the  USAF  by  other  organizations  in 
the  chain  of  command  and  by  the  USAF  on  subordinate  organizations,  specifically 
the  USAF  Systems  Command. 

The  Quality  and  Reliability  Assurances  (Q&RA)  policies  are  responsive 
to  the  Executive  Office  of  the  President  as  established  in  the  Office  of 
Management  and  Budget  (0MB)  Circular  A-109  dated  April  5,  1976.  It  is 
important  to  note  that  this  circular  was  not  the  driving  force  which  caused 
the  Department  of  Defense  and  the  U.S.  Air  Force  to  initiate  actions  to 
control  Q&RA  as  the  OSD  and  the  military  departments  had  recognized  the 
importance  of  quality  characteristics  of  products  several  years  before.  0MB 
Circular  A-109  did  signal  a  new  high  level  of  government  interest  in  weapon 
system  acquisition  and  identified  the  acquisition  of  major  systems  as  "one  of 
the  most  crucial  and  expensive  activities  performed  to  meet  national  needs. 

The  circular  identifies  performance/demonstration/test  characteristics,  all 
closely  associated  with  Q&RA,  along  with  others  that  require  attention  by  the 
program  manager. 

B.  POLICY  ORIGINS 

A  perception  of  quality  depends  on  the  viewpoint  and  interest  of 
the  observer.  Likewise,  quality  policy  is  obviously  affected  by  the  views 
—of  those  who  write  it.  It  is  useful  then  to  identify  at  the  outset  the  various 
organizations  which,  at  various  stages  in  the  system's  life  cycle,  have 
an  interest  which  waxes  and  wanes  depending  on  their  individual  role  in  any 
particular  system's  life  cycle.  Figure  I I -1  below  shows  the  principal 
observers  of  system  quality. 

There  would  be  little  significance  in  describing  in  detail  the 
perspectives  of  all  of  the  organizations  involved  other  than  to  say  that  the 
issues  that  receive  the  most  attention  are  system  cost  and  overall  performance 
goals.  For  the  purposes  of  this  discussion,  cost  will  be  discarded  and  per¬ 
formance  remains  as  an  overall  quality  surrogate  or  synonym.  Performance 
requirements  are  ultimately  defined  in  specifications  or  similar  documents  and 
range  from  broad  system  requirements  such  as  speed,  range,  overall  vehicle 
MTBF  to  specifications  for  elements  of  subsystems,  e.g.,  UHF  transmitters. 

The  Chief  Executive  and  the  Congress  have  a  responsibil ity  for 
allocation  of  national  resources  and  for  the  defense  posture  of  the  nation. 

The  interest  of  these  principal  elements  at  the  highest  levels  guarantees 
that  military  personnel  and  departmental  civilian  employees  will  have  a 
continuing  interest  in  how  quality  (in  terms  of  performance,  including 
support  costs),  is  specified  and  how  well  the  objectives  are  being  achieved. 

The  in-plant  quality  assurance  personnel  interface  daily  with  the 

contractors.  They  see  quality  primarily  in  terms  of  product  conformance 

and  their  view  is  based  on  production  perceptions.  Logistics  support  personnel 
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FIGURE  II-1 


SYSTEM  QUALITY  OBSERVERS 


including  reliability  and  maintainability  specialists  represent  another  range 
of  interest  and  are  concerned  with  the  frequency  and  costs  of  repairs  which 
are  fuctions  dependent  on  product  quality. 

On  the  contractor  s  s1de,  it  ''s  evident  that  the  spectrum  of  observers 
is  much  smaller.  Their  span  of  interest,  control  and  perspective  may  be  briefly 
intensive  then  may  erode  quickly  with  time  and  other  projects.  However,  quality 
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is  vital  to  their  survival  which  depends  on  their  producing  systems  that  are 
accepted  by  the  contracting  officers.  Contractors  have  an  inherent  patriotic 
interest  in  their  defense  equipment  performance  as  do  government  employees. 
There  are  other  fiscal  motivations  as  well  since  if  government  quality 
personnel  do  not  accept  their  system,  reimbursement  will  not  be  forthcoming 
and  the  firm  may  ultimately  go  out  of  existence. 

C.  GENERAL  POLICIES  FOR  THE  SPECIFICATION  AND  ATTAINMENT  OF  QUALITY 

1 .  Qua! ity 

Quality  is  defined  as  "the  composite  of  material  attributes 
including  performance,  features,  and  characteristics  of  a  product  or  service 
to  satisfy  a  given  need."4  By  this  definition,  product  quality  includes 
reliability  as  one  of  the  specified  attributes,  features,  or  characteristics 
of  a  product . 

Product  quality  is  specified  when  a  technical  description  of  the 
product  is  completed.  The  technical  description  may  take  the  form  of  a 
military  specification,  a  commercial  catalog  description,  company  part 
number  or  other  description.  The  product  description  may  also  contain  a 
modifier  such  as  grade  level.  It  may  also  simply  consist  of  a  description 
of  performance  goals  which  must  be  met. 

In  the  case  of  major  systems,  the  specification  of  quality  begins 
in  the  pre-conceptual  phase  of  the  life  cycle  by  a  determination  of  what  the 
performance  parameters  (including  reliability)  must  be.  In  the  initial 
specification  of  requirements,  performance  is  negotiated  and  traded  off  between 
minimum  operational  requirements  and  life  cycle  system  costs  throughout  the 
various  phases  of  the  life  cycle  until,  in  the  full-scale  production  phase 
they  are  firmly  established  as  technical  requirements  that  must  be  met. 

These  determinations  of  system  performance  requirements  are  made  by 

the  user  and  developing  command.  Deviation  from  these  requirements  must  be 

approved  by  the  appropriate  operating  command. 

The  quality  assurance  function  as  described  in  policy  documents 
is  principally  to  define  specifications,  standards,  inspections,  tests, 
training  and  certification  requirements  and  evaluations  required  to  control 
quality.  It  is  also  a  quality  assurance  function  to  identify  new  developments 
in  products  or  processes  that  will  necessitate  advanced  planning  of  quality 
assurance  activities. 

Other  specifically  identified  tasks  in  system  development  include 
the  quality  assessments  to  be  performed  before  the  end  of  the  demonstration 
and  validation  ohase  and  before  the  end  of  the  full-scale  engineering  develop¬ 
ment  phase.  The  purpose  of  the  assessments  are  essentially  to  assure  quality 
characteri sties  are  identified  and  quantified  and  to  assure  there  are  plans 
for  demonstration  of  conformance  to  quality  requirements. 

While  most  of  the  activity  is  related  to  administrative  controls, 
the  final  result  of  all  of  these  actions  is  the  act  of  acceptance.  Quality 
assurance  personnel  who  normally  perform  the  acceptance  function  signify  by 
this  action  that  all  requirements,  including  performance  characteristics, 
have  been  complied  with. 


4 


DoDD  4155.1,  Quality  Program 
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2 .  Rel iabil ity 


Reliability  may  be  described  as  continuing  performance  (as 
specified)  over  a  predetermined  period  of  time.  The  official  definition, 
is  "the  probability  that  an  item  will  perform  its  intended  function  for 
a  specified  interval  under  stated  conditions 5 

The  specification  of  essential  reliability  characteristics  starts 
with  the  Mission  Area  Analysis  (MAA)  performed  during  the  pre-conceptual 
phase.  Reliability  deficiencies  of  mission-related  operational  systems  are 
noted  then  and  the  essential  reliability  characteristics  of  the  new  system 
under  consideration  are  included  in  the  Statement  of  Need  (SON). 

During  the  conceptual  phase,  reliability  mission  needs  identified 
in  the  SON  are  translated  into  specific  values  relatable  to  program  management. 
The  terms  are  included  in  the  system  performance  description  and  in  operational 
and  maintenance  concepts.  During  this  phase,  the  reliability  goals  are  liable 
to  tradeoff  analysis  and  revision  in  conjunction  with  other  system  parameters. 

The  reliability  goals  of  the  conceptual  phase  are  transformed  into 
design  goals  which  are  pursued  during  the  demonstration  and  validation  phase. 
During  this  phase,  prototype  testing  is  performed  and  environmental  stress 
information  is  provided  for  further  design  deliberations.  Formal  reliability 
demonstrations  are  not  normally  performed  although  planning  for  reliability 
testing  in  later  phases  does  begin  then.  Effectiveness  or  cost  tradeoff  studies 
may  be  performed  as  a  result  of  updated  operational  or  maintenance  concepts. 
Ultimately,  the  testing  feedback  and  tradeoff  efforts  are  expected  to  result 
in  mature  operational  reliability  thresholds  and  goals  and  in  reliability 
requirements  for  inclusion  in  the  Full-Scale  Engineering  Development  ( FSED) 
phase.  The  program  manager  must  also  prepare  a  reliability  management  plan. 

The  contract  issued  for  FSED  must  have  quantitative  reliability 
requirements  included  in  it  and  the  requirement  must  be  demonstrated  in  an 
operational  test  and  evaluation  (OT&E)  environment  prior  to  the  start  of  full- 
scale  production.  The  reliability  management  plan  must  be  updated  to  support 
OT&E  activities  and  deployment.  By  the  end  of  this  phase,  reliability  require¬ 
ments  complete  with  associated  confidence  must,  if  possible,  be  developed. 

These  firm  reliability  requirements  can  then  be  incorporated  into  a  production 
contract . 


During  production,  testing  is  performed  periodically  to  ensure  that 
reliability  values  previously  demonstrated  have  not  been  degraded.  If  there 
is  evidence  that  reliability  requirements  are  not  being  met,  acceptance  will 
be  withheld  pending  satisfactory  compliance. 

D.  POLICY  DOCUMENTS 

This  section  describes  several  policy  documents  which  are  used 
either  singularity  or  in  combination  within  the  Air  Force  contracting  and 
acquisition  process  to  define,  specify  and  manage  quality  and  reliability 
requirements  for  products  and  systems  acquired  by  the  Air  Force. 


s  MIL-STD  721 B,  Definition  of  Effectiveness  Terms  for  Reliability, 
Maintainability,  Human  Factors  and  Safety 
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1 .  Office  of  Management  and  Budget  (0MB) 

Circular  A- 109  was  followed  by  OFPP  pamphlet  No.  1  published  by  the 
Office  of  Federal  Procurement  dated  August  1976  which  was  prepared  to  further 
describe  the  intent  and  application  of  A-109.  In  turn,  the  existing  DoD 
Directives  5000.1  (Major  System  Acquisitions)  and  5000.2  (Major  System 
Acquisition  Process)  were  revised  to  assure  consonance  with  the  0MB  policy. 

2 .  Office  of.  the  Secretary  of  Defense  (OSD) 

a .  Overview  of  Principal  Directives 

While  these  documents  contain  the  basic  system  acquisition 
guidance  from  the  OSD,  the  fundamental  quality  assurance  policy  is  contained 
in  DoDD  4155.1,  Quality  Program  and  in  the  Defense  Acquisition  Regulation 
(DAR).  As  in  the  cases  of  DoDD's  5000.1  and  5000.2,  4155.1  has  been  revised 
to  more  directly  reflect  some  of  the  system  strategy  concepts  embodied  in 
Circular  A-109. 

b.  DoD  Directives  5000.1,  5000.2,  5000.3  and  4155.1 

Of  these  documents,  DoDD  4155.1  treats  quality  assurance 
management  in  detail,  while  5000.1  and  5000.2  treat  product  quality  and 
reliability  only  in  very  broad  terms.  DoDD  5000.3,  Test  and  Evaluation, 
establishes  the  DoD  policy  for  the  conduct  of  the  test  and  evaluation 
during  the  acquisition  of  Defense  systems.  At  this  stage  in  the  regulatory 
process,  guidance  is  too  broad  to  include  specific  instructions.  The  general 
guidance  provided  is  that  major  systems  shall  be  developed  in  a  structured 
cost-controlled  process  in  which  a  series  of  pre-designated  program  issues 
must  be  addressed  in  making  recommendations  to  the  Secretary  of  Defense  to 
proceed  or  not  to  proceed  with  development  of  the  system.  Major  systems  are 
denned  as  those  involving  or  anticipating  a  cost  of  $75  million  dollars  in 
research,  development,  test  and  evaluation  (RDT&E)  or  $300  million  dollars 
in  production. 


c.  DoDD  4155.1 

DoDD  4155.1  (Quality  Program)  provides  a  broad  charter  of 
policy  and  objectives.  In  essence,  it  establishes  the  requirements  for  each 
of  the  DoD  components  to  develop  a  cost-effective  quality  program  which 
will  assure  that  DoD  material  both  conforms  to  specified  requirements  and 
provides  user  satisfaction. 

There  is  no  OSD  counterpart  directive  that  provides  policy  and/ 
or  objectives  in  the  related  areas  of  reliability,  maintainability  and  other 
similar  disciplines. 

While  DoDD  4155.1  is  a  very  broad  policy  document,  Paragraph 
C-6  is  concerned  specifically  with  the  system  acquisition  process  and  the 
product  quality  issues  alluded  to  in  DoDD's  5000.1  and  5000.2.  System 
program  managers  are  assigned  "responsibility  and  accountability"  for  the 
"quality,  reliability  and  maintainability"  of  their  products.  The  following 
system-related  requirements  are  identified  in  Paragraph  C-6: 
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o  Quality  characteristics  be  specified  and 
designed  into  the  product; 

o  Characteristics  be  quantified  whenever  possible; 

o  Critical  application  items  are  identified  and 
control  1 ed ; 

o  Quality  and  technical  requirements  be  achieved; 

o  Test  and  evaluation  be  performed  to  assure 
conformance; 

o  Design  reviews  and  independent  assessments  be 
performed  before  completion  of  each  milestone 
and  actions  be  taken  on  deficiencies  revealed; 
and 

o  A  copy  of  the  assessment  report  be  made  available 
to  the  head  of  the  DoD  component  concerned. 

3.  United  States  Air  Force 


a •  AFR  74-1,  Quality  Assurance  Program 

AFR  74-1  expands  on  the  broad  policies  and  specific  responsi¬ 
bilities  in  DoDD  4155.1.  It  establishes  clear  quality  assurance  management 
requirements  principally  in  the  areas  of  (1)  design  and  development,  (2) 
contracting,  (3)  contract  management  and  (4)  logistic  support,  and  generally 
sets  forth  USAF  quality  assurance  policy.  This  is  somewhat  of  a  variation 
from  DoDD  4155.1  which  is  written  in  terms  of  policy  objectives  and  responsi¬ 
bilities  without  regard  to  differing  management  functions  based  on  mission 
responsibil ities. 

Insofar  as  the  DoDD  4155.1,  Paragraph  C  system  requirements  are 
concerned,  they  are  restated  in  AFR  74-1  and  imposed  on  AFSC  in  Paragraph  3-D. 
Specifically,  one.  assessment  to  assure  the  objectives  have  been  achieved  is 
to  be  made  before  the  end  of  the  Demonstration  and  Validation  phase  and  one 
before  the  end  of  the  Full-Scale  Development  phase. 

In  summary,  AFR  74-1  is  primarily  concerned  with  management 
objectives  and  roles.  As  a  consequence,  the  quality  assurance  management 
system  concepts  and  approaches  to  controlling  product  quality  provides  limited 
technical  product  performance  oriented  instructions.  This  is  in  some  contrast 
with  AFR  80-5  in  which  product  performance  is  the  predominant  consideration. 

At  the  command  levels,  it  is  clearly  established  that  each 
acquisition,  modification  or  integration  program  must  have  a  program  manager 
with  specific  hardware  oriented  R&M  functions  to  perform. 

b.  AFR  80-5,  Air  Force  Reliability  and  Maintainability  Program 

AFR  80-5  outlines  the  policy  for  managing  the  Air  Force 
reliability  and  maintainability  program  for  systems,  subsystems,  equipment 
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and  munitions.  It  applies  to  all  Air  Force  activities  that  manage  such 
equipment  in  any  phase  of  the  life  cycle. 

This  directive,  as  AFR  74-1,  provides  comprehensive  management 
instructions  designed  for  the  Command  level,  e.g.,  Systems  Command/Logistics 
Command.  While  it  is  similar  in  that  it  describes  a  total  Air  Force  program, 
it  is  more  specific  and  more  demanding  in  management  requirements  and  in 
hardware  specificity.  . 

In  management  terms,  Air  Force  headquarter's  responsibilities 
are  outlined  in  AFR  80-5  from  both  a  technical  and  a  policy  viewpoint.  From 
a  technical  viewpoint , . i t  is  required  that  the  AF  headquarter's  element  assure 
that  R&M  or  at  least  mission  critical  parameters  be  included  in  each  Program 
Management  Procurement  (PMP).  It  is  also  required  that  a  program  element  for 
general  R&M  technology  improvement,  engineering  application,  and  control  be 
maintained.  From  a  management  viewpoint,  policy  is  established  and  offices 
of  primary  responsibility  assigned.  In  addition,  responsibility  for  an  AF 
wide  R&M  education  and  training  is  established  and  waivers  for  program 
exclusions  are  controlled. 

In  summary,  AFR  80-5  contains  a  significant  amount  of  specificity 
covering  equipment  in  R&M  terms.  It  provides  R&M  related  actions  in  some 
detail  that  are  to  be  performed  in  each  life  cycle  phase.  It  incorporates 
specific  R&M  performance  reports  under  established  Reports  Control  Symbols. 

c .  AFCMDR  74-1,  Procurement  Quality  Assurance  Program 

Air  Force  Contract  Management  Division  Regulation  (AFCMDR)  74-1 
establishes  policies  and  procedures  for  use  by  CMD  personnel  in  those  major 
facilities  that  have  not  bppn  assigned  to  the  Defense  Logistics  Agency  for 
administration  and  have  been  retained  for  administration  by  the  USAF.  There 
are  other  documents  which  have  an  impact  on  its  usage  such  as  MIL-STD  1520 
and  MIL-STD  1535.  However,  it  provides  a  view  of  the  philosophical  concepts 
which  guides  USAF  in-plant  quality  assurance  activities. 

AFCMDR  74-1  has  been  in  existence  in  various  forms  for  several 
years.  It  embodies  the  Contractor  Responsibility  Concept  and  is  predicated 
on  assuring  that  the  contractor  has  a  comprehensive  quality  program  and  that 
conforming  material  only  is  offered  for  acceptance.  The  regulation  has  much 
in  common  with  the  Defense  Logistics  Agency  and  Army  and  Navy  in-plant  quality 
assurance  documents. 

Plants  manufacturing  USAF  systems  have  traditionally  been  large 
facilities,  due  to  the  types  of  equipment  being  produced,  e.g.,  aircraft. 

It  was  reported  in  July  1977  that  contractor  quality  assurance  personnel  com¬ 
prised  over  14,000  persons  while  USAF  quality  assurance  numbered  1,185  persons. 
These  figures  are  considered  to  still  be  accurate  since  there  were  reported  to 
be  1,184  AFCMD  personnel  in  November  1979. 6  This  is  a  ratio  of  nearly  one  to 


AFCMDR  74-1  reflects  the  Contractor  Management  System  Evaluation 
Program  (CMSEP)  which  is  based  on  evaluating  contractor  management  as  a  system 
for  effectiveness,  adequacy  and  compliance.  While  superficial  views  of  documents 


6  Weiss,  Bernard  L.,  Observations  and  Recommendations  to  Enhance  Product  Quality 
in  the  Operational  Environment  (Quality  Horizons),  Air  Force  Systems  Command, 
1979 
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such  as  AFCMDR  74-1  may  be  misleading,  it  is  clear  that  it  is  based  on  general 
system  or  industrial  controls  with  less  emphasis  on  product  quality  in  the  sense 
of  product  reliability.  There  is  a  brief  treatment  of  quality  assurance  in 
system  acquisition  in  Chapter  2  which  is  largely  references.  Of  the  three 
elements  of  quality,  i .e. ,  quality  of  design,  quality  of  conformance,  quality 
of  contractor's  quality  assurance  system,  the  basic  thrust  of  the  AFCMDR  74-1 

is  to  deduce  quality  of  conformance  from  quality  of  system  tests,  while  quality 

of  design  is  essentially  untreated. 

AFCMDR  74-1  represents  the  most  recent  revision  to  what  has 
become  a  standard  concept,  consistent  with  DoD  policy,  of  how  to  assure  product 
quality  through  continuing  evaluation  of  the  contractor's  quality  assurance 
program.  This  approach,  which  to  some  extent  is  dictated  by  both  policy  and 
manpower  limitations,  results  in  limited  product  inspection  effort  and  fits 
into  the  overall  CMD  concepts  of  contract  administration. 

Defense  Acquisition  Regulations  (DoD) 

a .  General 

.  ^AR  is  di sti nqui shed  from  the  Directives  discussed  previously 

due _ to  its  prominence  in  the  procurement  hierarchy.  It  is  seen  as  more  than 

policy  since  it  also  contains  not  only  instructions  that  are  to  be  followed 
when  contracting  but  also  contains  specific  clauses,  forms,  etc.  that  are  to 
be  included  in  contracts  as  well .  The  DAR  establishes  the  general  procurement 
arena  in  which  major  systems,  as  well  as  all  other  military  procurements 
take  place. 


_  .  The  purpose  of  the  DAR  is  to  establish,  "for  the  Department  of 

Defense,  uniform  policies  and  procedures  relating  to  the  procurement  of  supplies 
and  services  under  the  authority  of  Chapter  137,  Title  10  of  the  United  States  ' 
Code,  or  under  other  statutory  authority."7  The  net  result  to  be  found  in  the 
DAR  are  the  policies  and  procedures  that  personnel  follow  who  are  involved  in 
the  procurement  of  supplies  including  the  administration  of  contracts.  The 
DAR  establ i shes  the  tone  and  perspective  of  the  DoD  and  Congress  across  the 
entire  spectrum  of  purchases  made  ranging  from  personal  troop  support  equipment 
to  major  weapons  systems. 

Also  included  in  the  DAR  are  necessary  presumptions  about  the 
DoD  market  place.  These  include  assumptions  of  the  market  insofar  as  what  will 
make  DoD  procurements  attractive  to  producers  and  what  contract  characteristics 
will  cause  those  firms  to  produce  material  on  schedule  at  the  agreed  upon  cost 
and  in  conformance  with  all  requirements. 

In  order  to  evolve  a  cohesive,  effective  and  universal  procurement 
approach,  the  DAR  identifies  and  describes  several  forms  or  modes  of  contracts, 
e.g.,  Fixed  Price,  Incentive,  etc.  that  will  be  used  in  various  contracting 
situations,  e.g..  Research  and  Development  and  Production.  For  the  established 
framework  of  contract  types,  specific  clauses  are  included  which  establish 
binding  contractual  terms  and  conditions.  The  DAR  also  describes  various 
government  roles  and  responsibilities  inherent  in  the  contract  such  as  identifying 
Contract  Administration  versus  Procurement  functions  and  provides  instructions 
as  to  what  type  of  clause  is  to  go  in  what  type  of  contract. 


DAR  1-101 
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For  the  purposes  of  the  investigation  into  contracting  method¬ 
ology  related  to  product  quality,  there  is  one  major  assumption  contained  in 
the  DAR ,  Paragraph  3-401 (a)  and  Paragraph  3-808.1.  This  assumption  is  that 
the  profit  made  on  a  particular  contract  will  be  reflected  in  the  quality  of 
material  delivered. 

b .  The  Role  of  Profit 

(1)  DAR  3-40l(b) (1 ) 

"Profit,  generally,  is  the  basic  motive  of  business 
enterprise.  Doth  the  government  and  its  defense  contractors 
should  be  concerned  with  harnessing  this  motive  to  work  for 
the  effective  and  economical  contract  performance  required 
in  the  interest  of  national  defense.  To  this  end,  the  parties 
should  seek  to  negotiate  and  use  the  contract  type  best 
calculated  to  stimulate  outstanding  performance.  The 
objective  should  be  to  insure  that  outstandingly  effective 
and  economical  performance  is  met  by  high  profits,  mediocre 
performance  by  mediocre  profits,  and  poor  performance  by  low 
profits  or  losses.  The  proper  application  of  these  objectives 
on  a  contract  by  contract  basis  should  normally  result  in  a 
range  of  profit  rates." 

(2)  DAR  3-808. 1(a) 

This  general  perception  of  the  nature  of  business, 
is  further  amplified  for  the  specific  case  of  cost  reimbursement  contracts 
in  the  DAR,  Paragraph  3-808.1  as  follows: 

3-808  Profit,  Including  Fees  Under  Cost-Reimbursement 
Type  Contracts 

3-808.1  Po 1 i cy 

(a)  General  "It  is  the  policy  of  the  Department 
of  Defense  to  utilize  profit  to  stimulate  contract  per¬ 
formance.  Profit  generally  is  the  basic  motive  of  business 
enterprise.  The  Government  and  defense  contractors  should 
be  concerned  with  harnessing  this  motive  to  work  for  more 
effective  and  economical  contract  performance.  Negotiation 
of  very  low  profits,  the  use  of  historical  averages,  or  the 
automatic  application  of  a  pre-determined  percentage  to  the 
total  estimated  cost  of  a  product,  does  not  provide  the 
motivation  to  accomplish  such  performance^ 

While  it  is  not  stated  specifically,  tie  assumption  is  the 
norm  for  a  free  market  and  accordingly,  that  is  presumed  to  be  the  basic 
underlying  DoD  procurement  assumption.  _ 

There  are  several  types  of  contracts  defined  and  prescribed  in 
the  DAR.  In  all  cases,  it  is  assumed  that  quality  is  a  primary  contract  goal 
along  with  price  and  schedule  and  that  the  product  will  conform  to  requirements. 
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One  basic  DAR  contract  form  is  the  Firm,  Fixed  Price  contract.  It  is  suitable 
for  use  when  reasonably  definite  designs  or  performance  specifications  are 
available  and  a  fair  and  reasonable  price  can  be  determined.  However,  another 
type  of  contract  is  cost  reimbursement  which  provides  for  payment  of  allowable 
costs  to  the  contractor  and  may  also  allow  a  sliding  scale  of  profit  based  on 
some  predetermined  contract  goal.  One  type  of  such  contract  is  the  Cost  Plus 
Award  Fee8  which  provides  a  means  of  applying  incentives  to  contract  goals  which 
are  not  susceptible  to  finite  measurement,  e.g..  Contractors  Quality  Assurance 
Program.  It  is  suggested  that  contractors  could  be  encouraged  to  perform  their 
quality  programs  to  a  higher  degree  of  efficiency  if  specific  profits  were 
achievable  and. in  addition  to  those  negotiated  as  a  profit  base  originally. 

This  premise  will  be  evaluated  in  a  later  section  of  the  report. 

Insofar  as  procurement  is  concerned  there  is  a  clear  aura  of 
public  trust  responsibil ity ,  which  if  examined,  would  include  a  requirement 
that  the  government  agent/buyer  did  in  fact,  receive  what  it  was  that  had 
been  contracted  for  in  the  name  of  the  public  (government).  This  can  simply 
be  stated  as  an  inspection  function  which  is  the  genesis  of  DoD  quality 
assurance  programs. 

While  there  are  several  principal  sections  of  the  DAR  that  bear 
significance  when  considering  the  DoD  quality  assurance  program,  Section  XIV 
should  be  considered  as  basic. 

c .  Section  XIV  Procurement  Quality  Assurance 

This  section  prescribes  policies  and  procedures  to  assure 
that  supplies  and  services  procured  by  the  Department  of  Defense  conform  to 
the  quality  and  quantity  set  forth  in  the  contract,  and  for  the  acceptance 
functions  associated  therewith.  It  is  worth  noting  that  the  DAR  is  limited 
to  procurement  quality  assurance;  storage  or  maintenance  quality  assurance 
policy  is  not  included. 

Part  I  of  Section  XIV  describes  and  defines  in  broad  terms 
contractor  and  government  responsibilities  in  terms  of  quality  assurance. 

Since  this  part  is  not  for  incorporation  into  contracts,  this  guidance  is 
intended  for  government  personnel  and  establishes  no  contractual  obligations. 

The  part  contains  a  matrix  for  selection  of  one  of  the  levels  of  intensity 
of  management  described.  In  effect  then,  this  part  of  Section  XIV  is  the 
basis  for  providing  instructions  to  procurement  personnel  as  to  what  sort  of 
a  quality  assurance  management  program  to  include  in  a  contract. 

Part  2  of  Section  XIV  establishes  the  responsi bi 1 ity  of  the 
various  concerned  government  organizations  involved  in  the  procurement  process 
as  follows: 


o  The  activity  responsible  for  the  product 
technical  requirements  is  also  responsible 
for  prescribing  "inspection  testing  or  other 
contract  quality  requirements . " 

o  The  Purchasing  Office  is  responsible  for 
"contractually  formalizing  requirements  for 
qual ity." 
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o  The  Contract  Administration  Office  is 
responsible  for  performing'  government 
quality  assurance  actions  (described  in 
Section  XIV,  Part  4),  and  for  reporting 
'  to  the  purchasing  office  any  deficiencies 

in  design  or  technical  requirements  including 
"contract  quality  requirements." 

Part  3  of  Section  XIV,  Contract  Provisions,  identifies  clauses 
contained  in  Section  VII,  and  provides  guidance  as  to  where  procurement 
quality  assurance  actions  will  be  performed..  It  also  establishes  policy 
concerning  the  responsibil ity  for  the  product  acceptance  functions,  when 
and  where  the  product  acceptance  will  be  affected,  and  provides  general 
guidance  on  certificates  of  conformance  and  small  purchases. 

Most  importantly,  it  identifies  specific  contract  quality 
assurance  clauses  for  inclusion  as  contractual  provisions.  Some  of  these 
clauses  can  have  a  distinct  impact  on  the  performance  and  outcome  of  a  contract 
and  will  be  discussed  further  in  the  special  clauses  section  of  this  report. 

Fart  4  establishes  policies  and  procedures  for  the  performance 
of  the  Procurement  Quality  Assurance  task.  This  task  is  essentially  the  entire 
spectrum  of  determining  if  contractors  have  fulfilled  their  contractual 
obligation  in  terms  of  product  quality.  It  reinforces  the  responsibil ity 
concepts  in  Part  1,  particularly  as  they  relate  to  the  government  quality 
assurance  functions.  Details  are  provided  relative  to  planning  and  imple¬ 
menting  the  government  actions  in  the  contractor  facility  environment.  The 
following  actions  are  directed  to  be  performed  to  "determine  the  contractors 
compliance  with  the  contract  quality  requi rements, "  (14-403): 

o  Review  and  evaluation  of  the  contractor's 
inspection  procedure. 

o  Review  and  evaluation  of  the  contractor's 
selection,  calibration,  maintenance  and  use 
of  gauges,  measuring  and  test  equipment. 

o  Review  and  evaluation  of  the  contractor's 
quality  records. 

o  Performance  of  product  verification  by  the 
government . 

There  is  other  information  in  this  part  concerned  with  adminis¬ 
trative  considerations  of  manufacturing  material  and  government  actions  in 
subcontractor  facilities,  government  inspection  stamps,  and  shipment  of  supplies. 
These  considerations  are  outside  of  the  primary  concerns  of  this  study  and  will 
not  be  discussed.  The  same  observation  is  valid  for  Part  5,  as  well,  since  it  is 
concerned  with  special  commodities,  i.e.,  subsistence,  petroleum,  and  construction. 

Section  XIV  is  then  limited  to  the  procurement  quality  assurance 
process.  It  describes  the  various  types  of  contractor  quality  assurance 
management  options  that  are  available  and  provides  a  guide  for  selecting  one 
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for  a  particular  contract.  It  establishes  the  basic  DoO  concept  that  the 
contractor  is  responsible  for  controlling  product  quality  and  offering  only 
conforming  material  to  the  government  for  acceptance.  It  provides  that  the 
contractor  may  be  penalized  in  the  event  the  government  is  required  to  expend 
additional  effort  as  a  result  of  the  necessity  for  re-inspection  by  the 
government.  Section  XIV  basically  provides  intra-government  management 
guidance,  identifies  standard  inspection  approaches  and  is  built  on  the 
general  premise  of  product  conformance  to  established  contract  requirements. 


In  addition  to  Section  XIV,  quality  considerations  can  be 
found  in  numerous  places  in, the  DAR  where  opportunities  are  provided  for  the 
use  of  quality  information.  Examples  of  such  DAR  references  where  product 
quality  data  can  influence  procurement  actions  include  the  following: 


1 -31 9(d) (Vi ii ) (A) 

1  -322.1  (b)(3)(H) 

1 -326.4(b) ( i i i ) 

1-604.1 ( i i ) ( A )  and  (B) 
1-902 
1-1110 
3-401 (b)(2) 

3-407. 2(a)(2) 

3-801 .3 


3-902. 3(c)(i)(ii) 


Renegotiation 

Multi-Year  Procurement 

Breakout  Guidelines 

Causes  for  Debarment 

Responsible  Contractors 

Qualified  Products 

Role  of  Profit  (and  Quality) 

Contracts  with  Performance  Incentives 
Responsibility  of  Requirements  and 
Other  Logistics  Personnel  in  Purchasing 
Offices 
Make  or  Buy 


d.  Contractual  Clauses 


No  quality  assurance  emphasis  can  be  applied  in  a  contract 
situation,  unless  there  is  enforcing  contract  provisions  or  language.  The 
DAR  references  above  are  solely  for  the  guidance  of  government  procurement 
personnel.  Section  XIV  is  fundamentally  administrative  guidance  on  how  to 
manage  the  Procurement  Quality  Assurance  task  and  as  such  deals  to  a  signi¬ 
ficant  extent  with  intra-government  rules  and  interfaces.  It  does,  however, 
lead  to  the  selection  of  administrative  quality  assurance  programs  and  specific 
clauses  for  inclusion  into  contracts.  Due  to  the  very  wide  and  thorough 
understanding  of  some  of  the  administrative  management  programs,  e.g  , 

Standard  Form  32,  MIL-Q-9858A,  they  will  not  be  discussed  in  detail.’  However, 
the  power  inherent  in  some  other  of  the  clauses  does  make  them  worthv  of 
limited  discussion.  Table  1 1 - 1 T  at  the  end  of  this  section,  identifies 
numerous  clauses  that  relate  to  the  quality  function.  Some  of  the  more 
significant  ones  are  discussed  in  the  following  paragraphs. 

DP  Form  1155  (Provision  No.  1  and  Responsibility  for  Inspection  Clause) 

Those  procurements  that  do  not  exceed  $10,000  are  generally 
referred  to  in  the  DAR  as  "small  purchases."  Negotiated  purchases  of  supplies 
not  in  excess  of  $10,000  may  be  affected  by  the  utilization  of  the  DD  Form 
1155.  Under  certain  conditions  set  forth  in  DAR  14-308,  the  DD  1155  may  be 
used  exclusively  as  the  quality  assurance  clause.  In  this  case  the  government 
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may  rely  fully  on  the  contractor  to  use  whatever  quality  control  methods 
he  believes  to  be  appropriate  for  that  particular  product.  There  are 
several  general  provisions  in  this  purchasing  document,  however,  the  first 
provision  incorporates  the  responsibility  for  a  contractor  to  perform  the 
inspection  and  tests  necessary  to  substantiate  that  the  material  conforms 
to  contract  requirements.  In  some  cases,  federal  specifications  may  contain 
the  standard  Responsibility  For  Inspection  clause  (DAR  7-103.24)  and  may 
identify  specific  inspection  requirements  in  Section  4  of  the  specification 
as  well.  There  may  also  be  drawings  incorporated  in  the  contract  which 
identify  inspection  requirements  and/or  incorporate  the  Responsibility  for 
Inspection  clause.  The  use  of  the  Contractor  Responsibility  alone  is 
restricted  to  purchases  under  $10,000.  Guidance  for  the  inclusion  of  these 
two  clauses  is  contained  in  DAR  14-101.5. 

Standard  Inspection  Clause 

This  requirement  may  be  incorporated  into  contracts  for  commercial 
items  or  a  military  item  that  is  not  associated  with  a  military  operation.  It 
requires  that  an  inspection  system  be  maintained  which  is  acceptable  to  the 
government  but  does  not  describe  in  advance  the  essential  characteristics 
of  the  system.  It  does,  however,  require  the  maintenance  of  'inspection  records 
and  gives  the  government  access  to  those  records.  The  clause  (DAR  7-103.5) 
contains  other  provisions  including  but  not  limited  to  the  following: 

o  The  rights  of  the  government  and  the  contractor 
if  non-conforming  supplies  are  tendered. 

o  Administrative  instructions  related  to  the  govern¬ 
ment  inspection  function  and  the  responsibilities 
and  rights  of  the  contractor  and  the  government. 

o  Contractual  terms  and  conditions  surrounding  the 
acceptance  functions. 

Guidance  for  its  inclusion  in  contracts  are  contained  in  DAR  14-101. 5. C. 

The  inspection  clause  must  be  included  in  contracts  for  supplies 
which  exceed  $10,000.  There  are  three  variations  of  the  clause  which  are  to  be 
imposed  depending  on  the  contract  form.  The  three  contract  forms  are:  (1) 
fixed  price  supply,  (2)  fixed  price  incentive,  and  (3)  retroactive  price 
redetermination  after  completion.  Items  (2)  and  (3)  require  the  deletion  of  the 
paragraph  concerned  with  government  rights  in  the  event  non-conforming  supplies 
are  delivered  and,  in  its  place,  substitute  a  different  paragraph  more  suited  to 
incentive  or  price  redetermination  types  of  contracts.  The  Inspection  clause 
must  be  included  when  either  of  the  two  following  requirements  are  imposed. 

Inspection  System  Requirements 

The  DAR  Inspection  System  referred  to  in  Paragraph  14-303  is 
documented  in  the  Military  Specification  MIL-1-45208.  The  determination  of 
whether  or  not  to  incorporate  it  into  a  contract  is  provided  by  the  DAR 
guidance  in  DAR  14-101. 5. b.  Basically,  it  is  designed  to  be  used  when  it  is 
necessary  to  impose  on  the  contractor  a  management  system  which  will  assure 
selected  management  controls  will  be  used  to  control  the  process  of  inspecting 


24 


and  documentation,  measuring  and  test  equipment  process  controls,  and 
non-conforming  material.  The  above  include  the  principal  areas  that  are 
specified,  however,  there  are  other  functions  as  well  that  are  of  a  lesser 
magnitude  of  significance. 

Quality  Program  Requirements 

The  Quality  Program  Requirement  is  documented  in  Military 
Specification  MIl-Q-9858.  As  in  the  case  of  MIl-I-45208,  guidance  for  its 
inclusion  in  a  contract  is  contained  in  DAR  14-101. 5. b.  MIL-Q-9858  is 
another  quality  management  system  but  of  greater  scope  than  MIL-I-45208. 

It  causes  a  manufacturer  to  control  everything  in  MIL-I-45208  and  the  following 
as  well— work  instructions,  control  of  subcontractors  and  quality  related 
costs.  Control  of  subcontractors  was  included  in  MIL-I-45208,  in  that 
inspection  must  be  performed  on  subcontracted  supplies,  however,  the  MIL-Q- 
9858  requirement  enforces  significantly  greater  controls. 

There  are  provisions  and  specific  clauses  in  the  DAR  for  the 
inclusion  of  the  Inspection  System  and  Quality  program  in  the  various  types 
of  contracts  discussed  above.  It  should  also  be  noted  that  they  are  subject 
to  modifications  to  fit  a  peculiar  contract  situation.  The  practice  of 
modifying  MIL-Q-9858A  occurs  most  often  in  weapon  systems  and  space  system 
project  offices.  The  Space  and  Missile  System  Organization  has  issued  a 
directive,  SAMSO  Std.  73-58,  Quality  Assurance  Requirements  for  Space  and 
Missile  Systems.  Its  purpose  is  to  "delineate  and  interpret  the  requirements 
of  MIL-Q-9858  for  application  to  space  and  missile  system  procurements." 

Inspection  and  Correction  of  Defects  (Clause  7-202.5) 

This  clause  is  normally  used  in  cost  reimbursement  type 
research  and  development  contracts  where  the  primary  contract  objective 
is  the  delivery  of  end  items  other  than  designs,  drawings  or  reports.  It 
incorporates  certain  elements  that  make  it  of  particular  interest  to  personnel 
concerned  with  the  inspection  and  acception  function. 

It  establishes  or  reaffirms  the  government's  rights  to  inspect 
and  test  the  product  being  contracted  for  to  the  extent  practical at  all 
times  prior  to  acceptance.  It  requires  the  contractor  to  maintain  an 
inspection  system  acceptable  to  the  government  and  establishes  the  right 
to  inspect  in  a  subcontractor's  plant.  It  provides  for  expeditious  final 
government  inspection  and  acceptance  after  delivery. 

The  most  important  condition  established  in  this  clause,  however, 
is  the  right  of  the  government  to  require  correction  or  replacement  of  non- 
conforming  material  for  a  period  of  up  to  six  months  after  acceptance.  In 
the  event  the  contractor  "fails  to  proceed  with  reasonable  promptness  to 
perform  such  replacement  or  correction,"  the  government  has  the  right,  among 
other  alternatives,  to  "by  contract  or  otherwise  perform  such  replacement 
or  correction  and  charge  to  the  contractor  any  increased  cost."  There  are  other 
penalties  as  well  and  the  result  is  that  this  clause  is  a  very  potent  contract 
term  with  the  potential  for  causing  serious  repercussions  to  the  contractor. 
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Liquidated  Damages  Clause  (DAR  7-105.5) 


This  clause,  like  the  Inspection  and  Correction  of  Defects 
clause,  is  of  particular  interest  to  quality  assurance  personnel  as  it 
provides  a  powefrui  incentive  to  contractors  to  provide  material  that 
conforms  to  requirements.  In  the  event  an  inspector  does  not  accept 
material,  and  the  supplies  or  services  are  not  delivered  on  schedule, 
the  contractor  may  be  forced  to  pay  to  the  government  a  previously  agreed 
upon  amount  for  each  calendar  day  of  delay. 


This  clause  is  normally  used  in  fixed  price  supply  procure¬ 
ments  by  formal  advertising  and  negotiation.  It  is  used  when  a  slippage 
in  the  delivery  schedule  will  cause  the  government  to  suffer  damages  and 
the  extent  of  the  damage  will  be  difficult  to  prove.  When  used  it  will  be 
included  as  a  part  of  the  Default  clause  (DAR  7-103.11). 


Material  Inspection  and  Receiving  Report  (Clause  7-104.62) 


This  clause  is  incorporated  in  all  contracts  which  anticipate 
the  delivery  of  a  separate  and  distinct  object.  This  clause  causes  the 
contractor  to  prepare  the  DD  250  form  which  is  normally  signed  by  the  QAR 
signifying  compliance  with  all  requirements  and  acceptance  as  an  authorized 
government  representative. 

Title  and  Risk  of  Loss  (Clause  7-103.6) 


This  clause  is  used  in  fixed  price  supply  or  fixed  price 
research  and  development  contracts.  It  is  of  particular  concern  to 
quality  assurance  personnel  since  it  provides  that  unless  there  is  specific 
exclusion  elsewhere  in  the  contract,  title  to  the  material  passes  to  the 
government  "upon  formal  acceptance." 


When  an  individual  checks  the  acceptance  block  on  the  DD  250 
and  signs  as  the  authorized  government  representative,  title  for  the  material, 
device,  or  system  passes  to  the  government  regardless  of  when  or  where  the 
government  takes  physical  possession.  Subsequent  to  this  action,  the 
contractor  may  ask  for  payment  for  the  material  and  present  the  DD  250  as 
all  or  partial  evidence  of  conformance  with  contract  requirements.  Without 
this  action  the  contracting  office,  under  ordinary  circumstances,  may  not 
recognize  a  request  for  payment.  The  significance  of  this  signatory  act  and 
the  responsibility  associated  with  it  is  one  of  the  major  government  actions, 
especially  on  major  weapon  systems.  It  is  this  acceptance  act,  normally 
assigned  to  quality  assurance  personnel,  that  in  effect  closes  the  procure¬ 
ment  cycle  loop  making  the  quality  assurance  role  one  of  prime  importance  in 
the  procurement  process. 

E.  DISCUSSION  OF  QUALITY  AND  RELIABILITY  POLICIES  AND  PRACTICES 

From  a  quality  standpoint,  there  is  no  recognizable  positive 
motivational  policy.  There  is  a  penalty-type  motivation  in  the  sense  that 
if  a  DD  250  is  not  signed,  cash  flow  to  the  contractor  will  be  at  least 
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interrupted.  There  is  the  same  consideration  inherent  in  the  government 
in-plant  quality  assurance  program  in  that  penalties  can  be  assessed 
formally  or  informally  by  varying  the  quality  assurance  approach  to 
inspection.  There  are  some  positive  aspects  such  as  permitting  reduced 
sampling,  or  the  CAP  concept,  or  the  references  to  quality  vs.  profit  in 
the  DAR.  In  general,  however,  there  is  no  recognizable  and  significant 
positive  motivational  approaches  or  initiatives  in  the  quality  assurance 
program.  Neither  the  negative  or  the  positive  features  of  quality  policy 
are  organized  into  any  centralized  approach. 

Reliability  policy  has  emphasized  -  the  necessity  for  specifying 
measurable  reliability  requirements.  In  conjunction  with  this,  there  has 
been  innovative  contracting  approaches  developed  such  as  Life  Cycle  Costs, 
Reliability  Improvement  Warranty,  etc.  These  are  essentially  profit-based 
motivational  concepts  and  generally  relate  to  DAR  approaches  such  as  the 
performance  incentive  and  can  either  increase  or  decrease  profits  depending 
upon  performance.  As  in  the  case  of  product  quality  policy,  product 
reliability  policy  is  not  organized  into  any  centralized  cohesive  approach. 

From  a  broad  procurement  policy  standpoint,  there  is  at  least  a 
theoretical  basis  for  motivating  contractors  by  relating  profits  to 
performance.  There  are  opportunities  to  relate  product,  quality  to  such 
areas  as  source  selection.  There  are  contracting  approaches  such  as  the 
CPAF  or  performance  incentive  techniques  where  there  are  opportunities  to 
use  the  profit  motive. 

There  is  no  visible  motivational  aspects  of  policy  anywhere  that 
directly  relates  to  the  operator  class  of  contractor  personnel  for  either 
product  quality  or  product  reliability.  There  is  no  significant  language 
in  policy  that  encourages  contractors  to  initiate  motivational  programs. 
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TABLE  1 1 -1 


PRODUCT  QUALITY  RELATED  CLAUSES 


Cl ause 

Supply  Contract 

R&D  Contract 

Fixed  Price 

Cost  Reimburs. 

Fixed  Price 

Cost  Reimburs. 

Inspection 

7-103.5 

7-302.4 

Title  and  Risk  of  Loss 

7-103.6 

7.205.3 

7-302.28 

■  7-404.7 

Payments 

7-103.7 

1 

7-302.2 

Responsibility  for  Inspection 

7-103.24 

Rights  in  Data  and  Computer 
Software 

7-104.9 

7-204.9 

7-303.54 

7-403.49 

Government  Property 

7-104.24 

7-203.21 

7-303.7 

7-402.25 

Quality  Program 

7-104.28 

7-204.10 

7-303.15 

7-403.15 

Inspection  System 

7-104.33 

Progress  Payments 

7-104.35 

7-303.23 

Value  Engineering 

7-104.44 

7-204.32 

7-303.31 

7-403.27 

Mew  Material 

7-104.48 

7-403.30 

Production  Progress  Report 

7-104.51 

7-204.47 

First  Article  Approval 

7-104.55 

7-204.39 

7-303.39 

7-403.34 

I 


Table  II-l  (Cont'd) 


PRODUCT  QUALITY  RELATED  CLAUSES 


Clause 

Supply  Contract 

P&D  Contract 

Fixed  Price 

Cost  Reimburs. 

Fixed  Price 

Cost  Reimburs. 

Material  Inspection  and 
Receiving  Report 

7-101.62 

7-204.44 

7-303.13 

7-403,41 

Preservation  Packing  and 
Packing  Requirements 

7-104.67 

Marking  of  Shipments 

7-104.68 

Safety  Precautions  for 
Ammunition  and  Explosives 

7-104.79 

7-204.49 

7-303.20 

7-403.24 

Aircraft,  Missile  and  Space 
Vehicle  Accident  Reporting 
and  Investigation 

7-104.81 

7-204.36 

7-303.48 

7-403.45 

Engineering  Change  Proposals 

7-104.89 

7-204.56 

7-303.58 

7-403.53 

Liquidated  Damages  j 

. .  .  .......  1 

!  7-105.5 

1 

7-304.7 

Warranties  j 

|  .  ..  ..  _  .  . . 

7-105.7 

7-304.10 

....  ...  ,  I,,  ■  --  ...  .  .  j 

Inspection  of  Supplies  and 
Correction  of  Defects 

1 . 

i 

7-203.5 

7-402.5 

Order  of  Precedence 

7-204.40 

7-303.40 

7-403.35 

Standards  of  Work 

7-302.3 

7-402.4 

Make  or  Buy  Program 

7-204.20 

7-403.14 

Pi. 
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QUALITY  AND  RELIABILITY  MOTIVATING  FACTORS 


III.  QUALITY  AND  RELIABILITY  MOTIVATING  FACTORS 


A.  GENERAL 

It  is  an  obvious  truism  that  the  classic  business  objective  of 
the  industrial  firm  is  to  make  money.  A  business,  however,  is  a  complex 
enterprise  and  on  a  particular  contract,  profit  is  not  necessarily  the 
prime  objective.  Other  factors  may  impact  on  the  profit  goals  of  a  firm 
depending  on,  for  example,  the  product,  the  phase  of  the  corporate  life 

cycle,  the  market  structure  and  other  considerations  as  well. 

B.  MOTIVATION  OF  THE  FIRM 

Obviously  a  firm  does  not  spring  into  existance  as  a  viable, 
successful  enterprise.  Under  ordinary  circumstances  the  firm  must  begin 
with  only  a  future  caoability  to  provide  a  product  or  service.  As  the 
firm  progresses  from  its  beginnings  to  full  operation,  its  goals  and 
motivations  are  in  a  continuous  state  of  change.  These  motivations  of 
the  firm  can  be  considered  to  be  a  "need"  and  a  hierarchy  of  needs  for 
the  firm  can  be  developed  as  Maslow9 10  developed  such  a  hierarchy  for 
individuals.  The  hierarchy  of  needs  for  the  firm  and  the  firm's  emphasis 
on  them  constantly  shift  as  the  firm  moves  through  its  own  unique 
development  cycle.  These  needs  have  been  identified  as  survival,  profit, 
growth  in  market  share  and  prestige.1 15 

While  survival  is  a  primary  concern  of  all  firms,  it  is  the 
primary,  and  initially,  the  only  need  of  the  emerging  firm.  All  other 
needs  become  secondary  until  the  survival  stage  is  safely  completed.  The 
corporate  hierarchy  of  qoals  can  be  illustrated  as  in  Figure  III-l  below 
until  survival  is  assured. 


Figure  III-l 

Corporate  Hierarchy  of  Goals:  Survival  Phase 


9  Maslow,  A.  H.,  Motivation  and  Personality,  Harpers  &  Bros.,  New  York,  1954 

10  Oppedahl ,  Phillip  E.,  Understanding  Contractor  Motivation  and  Contract 
Incentives ,  Defense  Systems  Management  College  1977 
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Once  survival  is  assured  it  then  becomes  necessary  to  consolidate 
the  firm's  position  in  the  market  and  its  continuing  existance  by  returning 
a  profit  above  and  beyond  the  income  necessary  for  salaries,  machinery, 
real  estate,  etc.  In  the  hierarchy  of  corporate  goals  then,  with  survival 
assured,  the  firm's  motivations,  or  needs,  may  be  depicted  as  in  Figure  1 1 1-2 
with  profit  as  the  primary  goal  or  need. 

Figure  1 1 1 -2 

Corporate  Hierarchy  of  Goals:  Profit  Phase 


Profit  does  not  resolve  itself  into  a  simple  goal  to  maximize  due 
to  several  other  issues.  Profit  is  an  elusive  term  and  goes  beyond  simply 
earnings  per  share  (EPS).  EPS  is  an  end  product  resulting  from  a 
array  of  actions  that  are  subject  to  wide  fluctuations  which  may  result 
from  management  decisions,  increases  or  decreases  in  such  expenditures 
as  R  &  D,  advertising,  employee  training  programs  and  other  discretionary 
outlays.  However,  at  some  point  in  time,  the  firm's  management  process 
will  result  in  a  decision  that  profit  is  adequate  or  normal  for  its 
industry,  and  that  the  firm's  growth  should  now  become  the  overriding 
objective.  At  this  time,  the  hierarchy  of  needs  of  the  firm  will  appear 
as  in  Figure  1 1 1-3 . 

Figure  I I I -3 

Corporate  Hierarchy  of  Goals:  Growth  Phase 


SURVIVAL  PROFIT  GROWTH  MARKET  SHARE  PRESTIGE 
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As  the  corporate  goal  shifts  in  the  direction  of  increasing 
growth,  the  central  thrust  will  be  growth  in  sales  since  increasing 
sales  results  in  growth  in  the  firm's  competence  and  capabilities. 

As  the  individual  contracts  become  larger  or  more  numerous,  there  is 
an  increasing  capability  to  improve  equipment  facilities  and  manpower 
strength.  Thus,  consideration  may  be  given  to  improving  a  relative 
position  in  the  market  place  and  the  overriding  motive  becomes  that  of 
increasing  the  company's  market  share  as  shown  in  Figure  III-4. 

Figure  III -4 

Corporate  Hierarchy  of  Goals:'  Market  Share  Phase 


At  the  time  the  firm  becomes  a  major  force  in  its  own  market  there 
is  little  more  to  be  gained  unless  a  monopoly  is  to  be  achieved.  The  major 
automobile  manufacturers  are  classic  examples;  it  is  not  enough  to  be  one 
of  the  largest  firms  in  the  nation  with  a  major  share  of  the  market.  What 
then  remains  is  to  gain  prestige  supremacy  or  to  become  number  1.  When 
this  is  achieved  the  firm's  goals  in  order  of  dominance  will  be  as 
depicted  in  Figure  III-5. 

Figure  1 1 1-5 

Corporate  Hierarchy  of  Goals:  Prestige  Phase 


SURVIVAL 


PROFIT 


GROWTH 


MARKET  SHARE 


PRESTIGE 
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Quality  and  profit  and  share  of  the  market  all  bear  a  relation¬ 
ship  to  the  life  cycle  phase  of  the  firm.  It  has  been  shown  that,  on  the 
average.  Return  on  Investment  (ROI)  goes  up  with  increases  in  the  market 
share.11  Those  firms  with  market  shares  above  36%  earned  more  than  three 
times  as  much  'relative  to  investment  as  businesses  with  less  than  7%  share 
of  their  market.  This  relationship  is  illustrated  in  Figure  III-6. 

Figure  1 1 1-6 

Return  on  Investment  as  a  Function 


Insofar  as  quality  is  concerned,  the  same  investigation  found 
the  relationship  between  share  of  business  and  product  quality  to  be  as 
shown  in  Table  III-l  below: 


Schoeffler,  Sidney,  Robert  D.  Buzzell  and  Donald  F.  Heany,  Impact  of 
Strategic  Planning  on  Profit  Performance,  Harvard  Business  Review, 
(March-April ) ,  Boston,  1974 
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Table  III-l 


Quality  As  a  Function  of  Market  Share 


%  of  Business  With 
Products  Of: 

Market  Share  Under  12% 

12-26% 

Over  26% 

Inferior  Quality 

47% 

33% 

20% 

Average  Quality 

30% 

36% 

30% 

Superior  Quality 

23% 

31% 

50% 

These  statistics  confirm  that  market  share  is  an  important 
motivational  consideration  that  has  a  definite  relationship  to  product 
quality.  They  also  confirm  that  increasing  the  market  share  may  very  well 
be  a  part  of  the  firm's  overall  strategy  and,  in  turn,  result  in  an  improve¬ 
ment  in  quality.  The  determinations  on  any  individual  contract  may  well  be 
concerned  with  factors  that  go  beyond  the  immediate  profit  tactic.  The  need 
to  grow  before  market  share  can  be  increased  is  obvious  and  the  benefits  to 
be  gained  by  increasing  market  share  are  also  clear.  Profit  on  any  individual 
contract  can  become  a  secondary  objective. 

C.  MOTIVATION  OF  CONTRACTOR  PERSONNEL 

The  Defense  Department  procurement  budget  for  fiscal  year  1980 
is  over  35  billion  dollars.12  The  size  of  this  market  by  itself  is  massive 
enough  to  motivate  contractors  to  wish  to  enter  it.  Studies  of  the  charac¬ 
teristics  of  business  ^irms  over  the  years  has  shown  that  there  are  numerous 
motivations  that  are  assignable  to  the  objectives  of  the  firm.  There  are, 
as  well,  other  interacting  personal  motivations  that,  while  usually  supportive 
may  be  divergent  from  the  firm's  goals  or  needs,  ’ 

Some  contractor  personnel  view  profits  as  an  abstract  issue  and 
contract  performance  as  a  real  issue  while  another  category  sees  contract 
performance  as  an  abstract  issue  and  profits  as  the  real  issue.  The  term 
"contractor"  i n  the  DAR  relates  closely  to  an  abstract  view  of  profit 
although  it  is  primarily  concerned  with  achievement  of  performance  objectives. 

In  order  to  gain  an  overall  perspective  of  all  of  the  motivations,  contractor 
personnel  are  divided  into  corporate  management,  upper  middle  management, 
project  managers,  and  operators  as  follows. 

1 .  Corporate  Management 

Corporate  management  is  defined  primarily  as  those  individuals  who 
must  interact  between  the  owners  of  the  firm  and  those  personnel  who  are  actively 
involved  in  the  day-to-day  management  process.  In  some  cases,  the  same 
personnel  fulfill  both  roles,  especially  in  small  firms.  In  larger  firms, 
these  management  officials  have  another  role  other  than  that  of  acting  as  a 
link  between  stockholders  and  operators,  and  that  is  their  responsibility 
for  the  broader  social  responsibilities.  This  role  of  social  responsibility 


12  Department  of  Defense  Annual  Report,  Fiscal  Year  1980 
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is  best  fulfilled  by  assuring  the  continuing  vitality  of  the  firm  which 
in  turn  provides  profits,  keeps  employment  levels  up,  fulfills  responsi¬ 
bility  to  creditors  and  affects  stability  in  those  geographical  areas 
where  company  personnel  are  situated. 

Corporate  management's  personnel  concerns  consist  of  a  complex 
bundle  of  human  needs  such  as  self-actualization,  esteem  and  power.  They 
also  have  strong  economic  drives  coupled  with  the  psychological  benefits 
resulting  from  personal /corporate  achievements.  A  national  survey  of 
executive  compensation  has  shown  that  bonuses  for  the  three  top  executives 
of  major  corporations  are  closely  tied  to  the  previous  year's  earnings.13 

At  this  level  of  management  the  question  of  meeting  or  exceeding 
reliability  and  quality  (R&Q)  requirements  would  not  seem  to  be  a  major 
consideration .  These  officials  will  most  likely  be  much  more  concerned 
with  factors  such  as  return  on  investment  or  earnings  per  share.  A 
further  list,  although  probably  of  somewhat  less  consequence  would  include 
market  share,  R&D  expenditures,  marketing  expenditures  and  investment 
intensity,  i.e.,  ratio  of  total  investment  on  sales. 

In  short,  corporate  owners  primary  interests  are  generally 
centered  around  profit  factors  and  since  corporate  officials  usually  serve 
at  the  pleasure  of  owners,  profit  becomes  their  major  concern.  Other  factors, 
some  of  which  are  discussed  above,  are  also  of  prime  importance.  The  question 
of  reliability  and  quality,  particularly  for  a  firm  devoted  to  military  sales, 
is  not  likely  to  be  a  subject  of  prime  interest  at  the  corporate  level. 

2 .  Upper  Middle  Management 

The  upper  middle  level  of  management  includes  typically,  division 
managers,  profit  center  managers  and  other  senior  officials  who,  while  not 
upper  corporate  officials,  are  influential  in  the  formulation  of  policy  and 
operational  procedures  and  are  concerned  with  customer  satisfaction  and 
associated  profits.  They  have,  however,  numerous  other  concerns  which  must 
be  balanced  against  profits. 

These  managers  are  defined  as  those  upper  level  managers  who,  while 
they  are  concerned  with  implementing  the  highest  corporate  policy,  are  in  a 
management  mode  in  which  they  make  significant  operational  decisions.  This 
group  of  managers,  depending  on  the  size  of  the  company  and  the  government 
program,  may  deal  directly  with  company  program  managers  or  may  have  less 
senior  officials  interfacing  between  them  and  the  program  manager. 

These  managers  may  be  assumed  to  reflect  a  mixture  of  needs  of  the 
corporation  and  needs  of  the  individual  .  Corporate  hierarchial  needs  described 
earl ier  were  identified  as  survival,  profit,  growth,  market  share  and  prestige. 


13  Rappaport,  Alfred,  Executive  Incentives  vs.  Corporate  Growth.  Harvard 
Business  Review  (July-August)  Boston,  1978 
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in  that  order.  Clearly  survival  would  be  the  most  basic  of  the  need  hierarchy. 
However,  once  satisfied,  the  other  needs  will  change  in  rank  of  importance 
as  newly  emerging  needs  of  the  organization  are  met  to  varying  degrees. 

Maslow,  as  is  well  known,  developed  a  hierarchy  of  needs  (modified  by 
McGregor )  ,14for-indiv'duals  which  in  the  case  of  the  middle  manager  also 
represent  important  factors.  These  needs  may  be  translated  roughly  into 
survival  (subsistence  earnings),  continuing  employment,  social  or  group 
acceptance,  ego  and  self-fulfillment.  While  the  middle  manager  teeters  on 
the  balance  between  the  needs  of  the  firm  and  his  own  humanistic  needs  and 
attempts  to  satisfy  both  those  senior  officials  above  and  junior  officials 
and  employees  below,  he  can  identify  with  more  force  and  clarity  corporate 
management  concerns. 

3 .  Project  Managers 

For  the  purposes  of  this  investigation,  the  other  key  influential 
source  of  managerial  impact  is  considered  to  be  the  project  manager.  It  is 
recognized  there  may  be  supervisory  levels  between  this  individual  and  middle 
level  management  and  there  are  certainly  supervisory  personnel  who  are  junior. 

We  will  consider  these  more  junior  managers  in  other  parts  of  the  study. 

However,  they  have  a  much  smaller  impact  on  the  significant  elements  of  the 
management  process.  The  project  manager  has  a  more  singular  and  insulated 
view  of  what  he  is  expected  to  achieve  in  terms  of  equipment  performance.' 

His  personal  perceptions  of  success  are  closely  related  to  direct  job 
success,  i.e.,  equipment  produced  on  schedule  within  cost,  meeting  all 
requirements.  The  project  manager's  goals  are  what  might  be  termed  instant 
goals,  that  is,  success  in  meeting  the  goals  is  immediately  demonstrabl e 
as  opposed  to  long-term  corporate  goals  such  as  increasing  market  share  or 
waiting  to  see  if  an  R&D  expansion  will  become  a  profitable  venture,  or 
waiting  years  for  a  long-term  incentive  to  be  paid. 

4 .  Operators 

Operators  are  defined  as  those  individuals  who  do  not  have  a 
supervisory  role.  They  are  normally  thought  of  as  blue  collar  or  hourly 
employees.  Forthis  discussion,  however,  all  non-supervi sory  personnel  are 
within  this  definition  including  design  engineers,  test  operators,  reliability 
engineers,  inspectors,  machinists,  etc.  They  are  included  because  they  have 
a  major  influence  on  how  well  equipment  will  operate,  on  its  durability,  as 
well  as  simply  getting  through  the  in-plant  inspection/test  process. 

This  group  of  employees  have  limited  or  in  many  cases  no  contact 
with  the  management  personnel  described  earlier.  Their  rewards  and  remuneration 
are  usually  established  by  either  personnel  policies  or  by  union  negotiations 
with  management  or  both.  Except  for  a  very  small  group  in  the  operator  class, 
they  have  limited  knowledge  of  how  well  the  system  they  are  working  on  is 
performing.  Generally,  their  rewards  bear  no  relationship  to  its  success 
other  than  they  do  or  do  not  have  a  job. 

This  category  of  contract  personnel  have  been  subject  to  much  study 
and  examination,  in  particular,  the  blue  collar  subgroup.  In  the  weapon  system 
industry,  however,  many  personnel  with  engineering  degrees  work  in  what  approaches 
assembly  line  conditions.  The  hundreds  of  engineers  at  desks  or  drafting  tables 
in  a  large,  busy,  weapon  systems  contractor's  plant  illustrates  this  situation. 


14  McGregor,  D.,  The  Human  Side  of  Enterprise,  McGraw  Hill,  New  York  1960 
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The  needs  and  motivation  of  these  individuals  have  been  examined 
by  such  authorities  as  Frederick  W.  Taylor,  Frederick  Heryberg,  Douglas 
McGregor,  A.  H.  Maslow  and  others  and  there  is  little  to  be  gained  restudying 
the  motivational  factors  that  can  influence  their  input  on  guality  of  product. 

It  is  worthwhile,  however,  to  identify  them,  in  no  particular  rank  or  order, 
as  safety,  membership,  self-esteem,  status/prestige  and  sel f-actual i zation .  It 
is  also  notable  that  the  worker  has  been  held  to  be  accountable  for  20  per¬ 
cent  of  the  product  defects. The  costs  associated  with  this  proportion  of 
defects  is  certainly  a  problem  worthy  of  attention. 

The  DAR  has  taken  the  approach  that  profit  is  the  prime  motivating 
factor  and  has  designed  several  contracts  for  various  situations,  e.g.,  R&D, 
production,  which  allows  profit  to  vary  depending  on.  contractor  performance  or 
risk.  Past  performance  is  considered  to  include  product  performance  and 
delivery  as  a  profit  consideration  and  is  considered  in  the  source  selection 
process,.  Performance  has  been  deleted,  however,  as  a  consideration  in 
determining  profit  factors  under  cost  reimbursement  contracts.16  There  is 
no  consideration  in  the  DAR  to  motivation  of  any  particular  segment  of  per¬ 
sonnel  in  the  firm,  however,  it  appears  that  profits  do  go  to  corporate  and 
upper  management  personnel  in  the  form  o^  bonuses,  etc.  No  consideration  or 
guidance  is  given  in  the  DAR  which  relates  to  personnel  such  as  operators 
and  their  needs  as  described  by  Maslow  and  others. 

D.  PROFIT  AS  A  MOTIVATOR 

The  presumption  of  profit  as  the  basic  motive  of  industry  is 
undoubtedly  true  in  the  general  sense  since  in  the  open  market  the  firm  will 
not  survive  without  profit.  For  the  DoD,  however,  it  may  be  an  overly  simplistic 
approach  and  one  that  is  flawed  since  it  is  (1)  based  on  a  questionable  assumption 
of  operating  in  an  open  market,  and  (2)  while  it  is  a  general  assumption,  it 
is  applied  on  a  contract  by  contract  basis.  In  other  words,  it  is  a  general 
assumption  applied  to  an  instant  contract  situation. 

The  Logistics  Management  Institute  reported  that,  "There  is 
virtually  unanimous  agreement  among  managers  and  analysts  who  have  studied 
overall  contractor  motivation  that,  in  the  short  run,  contractor  management 
does  sacrifice  short-run  profit  on  defense  business  in  favor  of  achieving: 

o  company  growth 

o  increased  share  of  the  industry  market 

o  better  public  image 

o  organizational  prestige 

o  carry-over  benefits  to  commercial  business 
(commercial  spin-offs) 

o  greater  opportunity  for  follow-on  business,  or 

o  greater  shareholder  expectations  for  future  growth 
and  profit. 


15  Juran,  J.  M.,  Quality  Control  Handbook,  Third  Edition,  McGraw  Hill. 

New  York,  1 974  '  '  - 

16  Defense  Procurement  Circular  76-3,  September  1976 
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Profit  sacrifices  are  made  in  the  process  of  acquiring  a  contract  as  well  as 
in  the  process  of  executing  it.  While  there  is  a  practical  limit  on  the 
extent  to  which  current  profit  can  be  sacrificed,  that  limit  rarely  causes 
short-range  profit  to  dominate  management  decision  making,17 

Another  view  of  the  reasons  firms  enter  the  defense  market  was 
presented  by  Professors  Hunt,  Rubin  and  Perry,  who  after  surveying  27 
industrial  organizations  found  that  despite  the  constant  stressing  of  profit 
motivation  by  government  personnel,  contractor  personnel  saw  the  government 
market  as  the  only  one  in  which  they  could  satisfy  their  desire  to  be  involved 
in  high  technology  fields. 30  In  the  survey  the  following  reasons  were  cited 
for  entering  the  government  market,  in  descending  order  of  importance: 

Percentage  of 


Reasons  for  Entering  the  Government  Market  Respondents  Citing  Reason 

The  government  market  was  the  only  market  or  the 

only  sizable  market  for  the  product  the  organi-  26% 

zation  wanted  to  sell 

The  government  is,  if  not  the  only  market,  a  large 

market  affording  high  volume  sales  24% 

The  government  market  offers  a  chance  to  develop 

personnel  (both  managerial  and  technical)  and/or  22% 

potential  commercial  products 

Natural  interest  with  involvement  in 

technological  fields  20% 

That  the  government  is  a  low  risk  market  (implying 

either  that  it  is  safe  or  that  it  is  a  low  17% 

investment  market) 

That  it  affords  opportunities  for  profit  14% 

Patriotic  duty  10% 

That  it  affords  a  chance  for  market  diversity  7% 

Glamour  or  opportunities  for  image  building  6% 


Insofar  as  product  quality  is  concerned,  the  appended  INTEC  survey 
showed  profit  in  the  form  of  performance  incentive  or  award  fee,  to  rank  well 
below  the  quality  related,  contractually  required  programs,  e.g.,  quality, 
reliability  as  a  quality  motivator.  The  findings  of  the  investigators  cited 
and  others  not  referenced  were  supported  by  the  INTEC  survey  in  which  82  ' 


Logistic  Management  Institute,  An_Exami nation  of  the  Foundation,  of 
.Contracting,  Washington,  D.C.,  May  1968 - - - — "  °  incent1ve- 

18  Hunt,  Rubin  and  Perry,  Federal  Procurement:  A  Study  of  Some  Pertinent 
Properties,  Policies  and  Practices  of  a  Group  of  Business  Organizations. 

State  University  of  New  York  at  Buffalo,  March  1969.  Percentages  cited 

do  not  add  to  100%  because  of  multiple  responses. 
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respondents  answered  how  they  would  rank  the  impact  of  various  contractual 
devices  as  far  as  the  quality  of  material  was  concerned.  In  line  with  other 
findings,  profit  ranked  well  down  in  its  impact  as  compared  to  reliability 
demonstrations  or  quality  and  reliability  programs.  Profit  in  this  context 
was  discussed  in  terms  of  cost  plus  award  fee  or  performance  incentive  type 
of  contracts . 


As  a  general. observation,  profi tabil ity  is  not  a  compelling  element 
of  government  business  in  general  when  compared  to  commercial  business.  The 
Profit  76  study  showed  an  average  of  14.8%  profit  before  taxes  for  commercial 
business  compared  to  5.6%  for  government  business.  Figure  I I I - 7  illustrates 
the  Profit  76  findings. 


The  findings  of  other  researchers  that  profit  is  not  the  foremost 
factor  is ^supported  by  the  findings  of  the  INTEC  investigation  as  well.  While 
profit,  of  necessity,  must  be  a  consideration,  its  motivational  power  varies 
depending  upon  the  peculiar  position  of  the  firm  in  its  own  hierarchy  of  needs 
and  it  cannot  be  considered  a  dominant  factor  on  each  contract.  Stated  again, 
the  overall  necessity  for  business  to  make  a  profit  is  not  necessarily  valid 
for  each  contract  issued  by  the  OoO. 


E.  CONTRACTOR  MOTIVATION  VS.  CONTRACTING  TECHNIQUES 

In  the  interview  and  questionnaire  process  conducted  as  part  of 
this  study,  numerous  questions  were  asked  that  generally  related  motivation 
in  contractual  terms  to  product  quality.  The  complete  questionnaire  results 
are  found  in  Appendix  A  of  this  report.  However,  selected  findings  are 
reviewed  in  the  following  paragraphs  in  connection  with  contractor  reaction 
to  motivations  that  correspond  to  OoO  contracting  motivational  aspects  both 
positive  and  negative. 


1 .  Direct  Profit  and  Quality 

Figure  II 1-8  relates  the  conclusion  drawn  from  82  firms  who  were 
asked  by  INTEC  to  rank  the  OAR  clauses  on  contracting  approaches  in  terms  of 
their  impact  on  quality.  The  respondents  were  given  the  option  to  rank  them 
as:  (1)  negative  impact,  (2)  no  impact,  (3)  slightly  beneficial,  and  (4)  very 
beneficial.  A  weighted  analysis  resulted  in  ranking  them  in  the  relative 
orders  shown  in  the  figure. 


It  can  be  seen  that  management  systems,  e.g.,  Quality  Program, 
Reliability  Program,  and  Maintainability  Program  were  seen  as  having  a 
more  positive  effect  on  product  quality  than  were  those  clauses  that  are 
considered  to  be  motivational  in  nature.  It  can  also  be  seen  that  the 
penalty-oriented  or  negative  motivational  clauses  such  as  liquidated  damages 
have  little  benefit  as  well.  As  far  as  product  quality  and  motivational 
contracting  approaches  are  concerned,  this  data  shows  that  the  customary 
management  programs  are  perceived  by  the  respondents  as  more  effective 
than  either  positive  or  negative  motivational  approaches. 
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Figure  1 1 1 - 7 


PROFITABILITY  -  RETURN  ON  SALES 
PROFIT  BEFORE  TAXES/SALES 
5  YEAR  AVERAGE  BY  PRODUCT  GROUP 
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Figure  III -8 

Contractor  Ranking  of  DAR  Clauses  in  Terms  of  Impact  on  Quality 


When  the  respondents  were  asked  to  rate  financial  incentives,  e.g., 
profit,  as  a  management  tactic  to  assure  suppliers  provided  material  of  the 


requisite  quality  they  responded 

as  follows: 

No  Benefit 

Slight  Benefit 

Si gni ficant 
Benefit 

Best  Method 

No  Response 

17% 

46% 

27% 

4% 

5% 

When 

response  was: 

asked  the  same  question  about 

psychological  incentives,  the 

No  Benefit 

Slight  Benefit 

Significant 
Benefi t 

Best  Method 

No  Response 

23% 

48% 

16% 

1% 

1% 
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When  asked  to  rank  low  profit  as  a  problem  resulting  in  the 
production  of  non-conforming  material,  the  answers  were: 


Signi ficant 


No  Problem 

'  SI ight  Probl em 

Probl em 

Serious  Problem 

No  Response 

39% 

34% 

18% 

2% 

7% 

In  order  to  equate  low 

profit  as  a  factor,  respondents 

were 

also  asked  to 

rank  poor  specification  of  requirements  in  the  same  way. 

The  response  was: 

- 

Signi fi cant 

Serious' 

No  Problem 

Slight  Problem 

Probl em 

Probl em 

No  Response 

4% 

8% 

43% 

43% 

2% 

During  the  course  of  interviews  with 

government  personnel,  it  was 

suggested  that 

contracts  of  longer  duration  would  be  desirable  to  motivate 

employees  to  high  quality  workmanship.  That  question  was  responded  to  as 

follows : 

Significant 

No  Benefit 

Slight  Benefit 

Benefi t 

Best  Method 

No  Response 

17  % 

29% 

28% 

7% 

19% 

When 

special  bonuses  or 

cash  awards  were  considered,  the  answers  were 

Si gni ficant 

No  Benefit 

Slight  Benefit 

Benefi t 

Best  Method 

No  Response 

1 2% 

35% 

37% 

10% 

5% 

When 

increased  pay  was  - 

considered  the 

response  was : 

Significant 

No  Benefit 

SI ight  Benefi t 

Benefit 

Best  Method 

No  Response 

15% 

50% 

25% 

4% 

6% 

Fiscal  related  rewards  ’ 

were  equated  to  psychological  incentives. 

Official  recognition  was  considered,  the  answers  were: 

Signi ficant 

No  Benefit 

Slight  Benefit 

Benefi t 

Best  Method 

No  Response 

0% 

4% 

54% 

39% 

3% 

The  results  of  the  above  survey  indicates  that,  in  terms  of  product 
quality,  profit  for  the  firm  and  increased  financial  rewards  for  employees 
are  not  particularly  effective  motivators.  In  the  same  vein,  penalty  oriented 
contractual  clauses  are  shown  also  to  have  little  effect  on  product  quality. 
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2 .  Incentivised  Profit  vs.  Quality 

The  DAR  contains  quality  associated  options/clauses/approaches 
that  can  be  considered  motivational  in  terms  of  additional  income  by 
i ncorporation  of  incentive  fees  for  varying  levels  of  performance.  These 
can  be  added  to  anticipated  profit  or  sometimes  deducted  from  it  as -a 
consequence  of  achieving,  not  achieving  or  surpassing  performance  goals. 

Respondents  to  the  INTEC  questionnaire  were  asked  if  they  had 
experience  with  selected  incentive  type  approaches,  what  they  would 
expect  the  impact  to  be  on  quality  when  they  were  included  in  a  contract. 
The  methods  cited  were  Life  Cycle  Cost,  Failure-Free  Warranty,  Multi- 
Level  Acceptance  Sampling  Plans,  Performance  Incentive,  and  Cost  Plus 
Award  Fee  (CPAF).  The  responses  were  as  follows: 


Negative 

Impact 

No  Impact 

SI ightly 
Beneficial 

Very 

Beneficial 

No 

Experience 

50% 

Life  Cycle  Cost 

0% 

1 21 

26% 

12% 

Failure-Free 

Warranty 

0% 

12%. 

11% 

18% 

59% 

Multi-Level  Accep¬ 
tance  Sampling  Plan 

2% 

17% 

28% 

13% 

40% 

Perfo rmance 
Incentives 

o  of 

C  io 

10% 

17% 

32% 

39% 

CPAF 

0% 

11% 

18% 

16% 

55% 

The  above  data  indicates  that  there  is  some 
in  relation  to  product  quality  by  using  this  group  of 
can  directly  or  indirectly  lead  to  increasing,  and  in 
profit  margins.  The  data  also  gives  some  insight  into 
related  approaches.  No  response  was  interpreted  as  no 

benefit  to  be  derived 
clauses,  all  of  which 
some  cases  decreasing, 
the  usage  of  the  profit 
experience. 

There  are  no  "pure"  motivational  approaches  in  the  DAR  that  do  not 
relate  to  profit  in  some  form.  However,  two  motivational  techniques  that  have 
been  used,  or  at  least  encouraged  in  contracting  methodology  are  Zero  Defects 
and  the  Draft  Request  for  Proposal.  The  same  question  regarding  these  methods 
impact  on  quality  was  asked  and  the  responses  were  as  follows: 

Negative 

Impact 

No  Impact 

SI ightly 
Eeneficial 

Very 

Beneficial 

No 

Experience 

44% 

Zero  Defects 

4% 

18% 

23% 

11% 

Draft  RFP 

1% 

12% 

21% 

15% 

51% 

^  1s  dear  that  many  of  those  respondents  that  have  utilized  these 
methods  have  found  them  beneficial  and  equally  clear  that  they  are  not  extensively 
used.  J 
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The  intent  of  penalty  clauses  presumably  is  to  encourage  firms 
to  produce  material  of  the  requisite  quality.  Accordingly,  respondents 
were  asked  about  the  impact  on  quality  when  the  liquidated  damages  clause 
is  employed.  The  answers  were: 

Negative  Slightly  Very  No 

Impact  No  Impact  Beneficial  Beneficial  Experi ence 

Liquidated  ...  „„  , 

Damages  ®  J  ^4%  9%  0%  61% 

F.  CONTRACTUAL  MODES  AMD  CONTRACTOR  MOTIVATIONS 

In  considering  motivation  in  terms  of  Air  Force  equipment  products, 
it  may  be  useful  to  examine  the  system  procurement  process  and  review  how 
equipment  performance  goals  are  established  and  how  firms  respond  to  the 
contractual  mode.  For  this  example,  an  aircraft  case  will  be  utilized. 


In  the  system  procurement  process,  the  USAF  has  available 
considerable  latitude  in  deciding  how  to  interface  with  the  multiplicity  of 
contractors  that  will  necessarily  be  involved.  Once  the  contracting  office 
has  made  a  firm  decision  as  to  what  configuration  of  product  will  be  bought 
and  selected  the  prime  contractor,  there  are  still  numerous  secondary  level 
decisions  that  are  necessary.  For  example,  shall  equipment  to  be  integrated 
into  the  prime  system  be  selected  by  the  purchasing  office  and  be  provided 
by  associate  contractors  and  therefore  be  Government  Funded  Equipment  (GFE), 
or  shall  it  be  selected. by  the  prime,  or  integrating  contractor  and  provided 
by  subcontractors.  It  is  obvious  that  contractor  incentives  and  motivational 
considerations  are  affected  by  such  decisions.  Figures  III-9,  10  and  11 
illustrate  some  of  the  possible  contracting  modes.  The  prime  weapon  system 
in  the  example  is  that  of  aircraft. 


Figure  I I I -9 
Single  Prime  Contractor 


In  Case  #1,  Figure  I I 1-9 ,  the  prime  contractor  has  an  obvious 
need  to  assure  that  his  subcontractors  (all  suppliers)  are  successful  and 
that  he  is  successful  in  making  their  equipment  perform,  interface,  and 
operate  successfully  within  his  airframe  subsystem.  Subcontractors  also 
have  greater  pressure  to  perform  against  the  threat  of  being  eliminated 
as  a  supplier. 


Figure  III-10 
Integrating  Contractor 


In  Case  #2,  "igure  III-10,  the  integrating  contractor  is  typically 
an  aircraft  manufacturer  who  normally  builds  the  airframe  and  provides  some 
or  all  of  the  subsystems  and  integrates  them  into  the  airframe.  In  this 
example,  the  integrating  contractor's  desire  to  make  all  of  the  subsystems 
perrorm  together  will  be  altered  by  the  fact  that  the  government  selected  some 
of  the  subsystems.  Thus,  the  lines  of  responsibility  begin  to  blur  between  the 
systems  selected  by  the  government  and  those  selected  by  the  contractor.  The 
contractor  now  cannot  be  expected  to  assume  responsibility  for  Government 
furnished  property's  performance  and  must  be  reimbursed  if  it  can  be 
demonstrated  that  a  fault  in  Government  provided  equipment  requires  extra 
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effort  to  make  it  function  successfully.  The  responsibilities  of  associate 
contractors  selected  by  the  government  are  complex  and  careful  government 
management  is  in  order.  There  is  more  opportunity  to  escape  penalty  for 
non-performance  since  lines  of  responsibility  become  increasingly  blurred 
between  the  associate  contractor,  their  contracting  officer  and  the 
integrating  contractor. 


Figure  1 11-11 

Direct  Project  Management  Control 


Case  #3,  Figure  III-ll,  need  not  be  explored,  however,  since  it 
can  be  seen  that  it  simply  compounds  the  problems  of  Case  §2  and  increasingly 
exposes  the  program  manager  to  system  administration  and  management  problems. 
The  AFLC  (or  other  AF/ government  supplier)  may  also  exacerbate  the  problem 
to  an  even  greater  degree  due  to  the  additional  government  interfaces. 

Motivations  of  producers  and  users,  in  the  sense  of  prime  or 
integrating  contractors,  can  be  seen  in  the  above  cases  to  add  to  the  welter 
of  similar  and  diverging  needs  of  the  significant  tiers  of  management.  Added 
to  this  are  the  variations  in  motivational  perceptions  of  corporate  and  middle 
management  which  are  directly  affected  by  the  corporate  hierarchy  of  needs. 

In  a  system  procurement,  the  position  of  the  firm  on  the  ladder  of 
corporate  needs  will  result  in  a  differing  posture  by  the  corporate  and  middle 
management  executives  in  response  to  problems  being  encountered. 
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The  conclusions  then  that  can  be  drawn  from  the  mix  of  corporate 
and  individual  needs  impacting  on  every  system  procurement  are  (1)  that 
no  one  overall  government  strategy  can  be  applied  successfully  and  uniformly 
to  all  contracts  without  regard  to  the  hierarchial  needs  posture  of  the  firm 
at  the  time  of  negotiations ,  and  (2)  profit  should  not  be  assumed  to  be 
the  primary  contractor  motivation  on  every  contract. 
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ANALYSIS  OF  POLICY- AND  COMPATIBILITY  WITH  CONTRACTING  MOTIVATIONS 


IV.  ANALYSIS  OF  POLICY  AND  COMPATIBILITY  WITH  CONTRACTOR  MOTIVATIONS 


A.  OVERVIEW  OF  QUALITY  AND  RELIABILITY  POLICY 

1 .  Background 

Quality  policy  is  an  all  encompassing  term  used  repeatedly  in 
quality  assurance  personnel  discussions  and  in  the  quality  related  policy. 

In  comparison,  reliability  policy  is  a.  less  apparent  term.  Product  quality 
policy  is  also  less  apparent,  as  most  quality  policy  is  organizationally  based, 
particularly  in  the  DoD,  i  .e. ,  DAR  and  other  DoD  directives.  By  contrast, 
reliability  policy  is  principally  product  based  with  limited  organizational 
related  guidance.  There  is  a  clear  line  drawn  between  quality  and  reliability 
functions  at  all  levels. 

ihese  two  disciplines  generally  are  silent  as  far  as  recognition 
of  each  other  is  concerned  in  their  respective  directives.  Since  product 
quality  cannot  exclude  time  related  performance  characteristics,  e.g., 
reliability,  the  artificial  distinction  appears  to  serve  no  useful  purpose. 
Within  quality  assurance  organizations,  the  emphasis  on  quality  assurance 
organizational  matters  and  contractor  administrative  programs  may  have  a 
tendency  to  overshadow  product  Quality.  Another  general  characteristic 
of  "quality  policy"  is  the  loose  usage  of  the  term,  quality,  e.g.,  quality 
personnel,  contractor  quality  data,  quality  requirements,  contract  quality 
requirements,  quality  data,  etc.  This  proliferation  of  language  diffuses 
and  blurs  the  meaning  of  quality.  There  is  limited  guidance  dealing 
specifically  with  product  quality. 

The  USAF  has  traditionally  had  an  appreciation  of  the  complexities 
of  system  procurement  and  has  a  record  of  aggressively  developing  new  and 
innovative  methodology  intended  to  increase  the  yield  of  acceptable  versus 
unacceptable  equipment  produced  in  the  manufacturing  environment.  It  is 
obvious,  of  course,  that  the  greater  the  production  of  acceptable  material, 
the  better  the  cost  ratio  will  become  and  the  more  procurement  costs  will 
be  driven  in  the  direction  of  minimizing  costs. 

The  litany  of  techniques  used  over  the  years  includes  product 
inspection,  testing,  warranties  and  numerous  other  contracting  techniques. 
Traditionally,  military  procurement  methods  for  assuring  the  quality  of 
and  reliability  of  material  have  been  an  examination  before  payment, 
in  the  form  of  inspection  or  test,  with  a  penalty-based  system  of 
correction  when  defective  material  is  found  to  have  been  delivered. 

There  is  an  increasing  trend  in  the  direction  of  reward-oriented  warranties 
which  relate  extra  profits  to  field  performance  as  in  the  case  of  the 
Reliability  Improvement  Warranty  (RIW).  Incentive  fees  of  varying  types 
have  gone  through  intensive  cycles  of  high  and  low  usage  and  in  fact  go  back 
to  the  first  airplane  bought  from  the  Wright  Brothers.  Along  with  the 
innovative  approaches  have  been  strict  penalizing  contract  forms  such  as 
the  Correction  of  Defects  and  Constructive  Damage  clauses. 

The  reliability  and  quality  of  material  delivered  to  the  USAF 
clearly  has  an  impact  on  the  military  readiness  of  the  USAF  in  terms  of 
mission  effectiveness.  Reliability  and  quality  of  material  also  have  a 
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direct  bearing  on  procurement  demands  and  supply  activities;  effect 
operation  and  support  costs;  and  even  impact  transportation  requirements 
and  capabilities.  Furthermore,  during  the  procurement/production  phase 
of  systems  acquisitions,  procurement  costs  are  affected  by  scrap,  rework, 
repair  and  inspection  costs.  Any  costs  in  these  areas  cancel  expected 
improvements  in  supply  positions  and  reduce  the  availability  of  procurement 
funds.  Ultimately,  defective  and  unreliable  material  defers  the  entry  of 
new  systems  into  an  operational  posture  and  reduces  the  availability  of 
equipment  for  tactical  deployment. 

2 .  Confusion  About  Quality 

Product  quality  is  reviewed  by  all  levels  of  government  managers, 
ranging  from  the  inspector  to  the  0MB  and  Congress.  Most  public  discussion 
of  system  performance  occurs  when  the  system  exhibits  catastrophic  failures 
in  the  public  view.  Since  systems  that  perform  well  do  not  receive  equal 
notice,  poor  system  quality  is  a  common  public  perception. 

System  acquisition  phase  quality  frequently  becomes  an  issue  when 
Operational  Test  and  Evaluation  (OT&E)  organizations  decide  a  system  should 
not  be  fielded,  or  at  least  publicize  critical  test  findings.  The  ensuing 
disagreements  are  frequently  between  the  system  proponent,  the  Project  Office 
and  the  OT&E  group,  and  often  require  referee  decisions.  The  second  area 
where  system  quality  is  frequently  disparaged  is  early  in,  and  sometimes 
throughout  the  deployment  cycle  when  logistic  support  activities  find  the 
system  is  expensive  to  maintain.  In  the  third  case,  the  operational 
organizations  are  not  able  to  perform  desired  missions. 

At  the  same  times  these  issues  arise,  the  quality  assurance 
organization  that  accepted  the  equipment,  i.e.,  signed  the  DO  Form  250, 
is  frequently  suspected  of  not  performing  their  responsibility  properly. 

There  has  arisen  what  may  be  described  as  the  poor  quality  syndrome,  which 
becomes  a  vague  and  illusive  but  pervasive  complaint  about  "quality"  which 
is  dependent  upon  who  is  categorizing  the  system  as  poor  quality  and  what 
his  particular  definition  of  quality  is.  The  individual's  conception  of 
quality  leads  to  frequent  questions  based  on  the  role  and  perception  of 
those  evaluating  quality. 

The  quality  of  a  weapon  system  is  perceived  by  the  upper  levels 
of  military  and  government  management,  e.g.,  Hdqtrs.  USAF,  Office  of  the 
Secretary  of  Defense  and  Office  of  Management  and  Budget  as  a  function  of 
general  operational  characteristics  of  the  system. 

Using  an  old,  but  classic  example,  Bell  Telephone  Labs  decided  to 
use  an  existing  rocket  motor  for  the  NIKE  AJAX  ground-to-ai r  missile  system.19 
This  motor  used  nitric  acid  and  aniline  fuel  and  was  selected  in  place  of 
a  ram  jet  engine  which  offered  potentially  greater  range  and  operating 
economy.  The  decision  was  made  on  the  basis  that  the  rocket  represented 
the  current  state  of  the  art  and  was  known  to  be  feasible,  while  the  ram 
jet  required  further  research.  The  ultimate  quality  of  the  weapon  system 


_  / 

19  Bell  Telephone  Laboratories,  Inc.  Report,  AAGM:  A  Study  of  an  Anti- 
Aircraft  Guided  Missile  System,  July  15,  1945 
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then  was  reduced  as  a  result  of  a  technological  determination.  This 
demonstrates  what  may  be  called  quality  of  design,  which  relates  to  what 
may  be  termed  levels  or  grades  of  quality,  e.g.,  JAN  grade  electronic 
devices  vs.  commercial  grade. 

Personnel  who  are  responsible  for  determining  if  material  is 
acceptable  in  terms  of  contract  requirements  will  be  concerned  with 
assuring  that  the  product  under  examination  conforms  in  all  respects  to 
the  established  design.  Their  responsi bi 1 i ty  is  not  the  quality  of  design 
but,  instead,  with  assuring  that  the  product  meets  all  specified  requirements . 

As  discussed  in  other  parts  of  the  report,  quality  has  two  elements,  quality 
of  design  and  quality  of  conformance.  Design,  however,  is  not  normally 
examined  in  terms  of  excellence  or  in  other  words,  quality. 

There  are  many  truisms  associated  with  quality  which  represent 
the  conventional  wisdom  and  are  often  expounded;  the  following  are  examples; 

o  Quality  is  everyone's  business 

o  Good  quality  is  essential 

o  Good  quality  means  better  than  ordinary  quality 

o  Some  errors  o*  workmanship  are  inescapable  and 
excusabl e 

o  The  government's  job  is  to  specify  quality  and 
the  contractor's  job  is  to  deliver  that  quality 

o  There  are  levels  of  quality 

o  If  you  want  better  quality,  increase  the 
contractor's  profit 

These  perceptions  cloud  the  product  quality  question.  Regardless  of  merit, 
they  do  form  the  basis  of  many  discussions  about  product  quality  but  it  is 
obvious,  most  will  not  provide  a  reasonable  basis  for  attempting  to  determine 
why  there  are  quality  complaints. 

Assuming  that  quality  assurance  personnel  are  accepting  precisely 
as  is  specified,  then  if  there  are  valid  complaints  about  performance  or 
support  costs  caused  by  frequent,  unexpected  failures,  they  must  arise  from 
poor  specification  of  requirements,  or  a  poor  design.  The  function  of 
defining  quality  of  design  is  performed  essentially  by  middlemen  between  the 
inspector  and  the  designer.  Quality  complaints  provided  to  contract  adminis¬ 
tration  organizations  usually  originate  in  other  functional  organizations  in  which 
the  design  function  is  not  recognized  as  a  quality  function. 

3 .  Quality  Defined 

Quality  is  defined  by  the  Department  of  Defense  (DoD)  as  "the  composite 
of  material  attributes,  including  performance  features  and  characteristics , 
of  a  given  product  or  service  to  satisfy  a  given  need."  This  definition  is 
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concise  compared  to  the  Webster's  Dictionary  definition  which  offers  several 
possible  variations.  In  the  Webster's  definition,  however,  the  term 
“excellence"  does  appear.  The  DoD  definition  is  almost  hypothetical  in 
nature  and  a  transition  from  definition  to  product  is  difficult.  It  is 
likely  that  some  complaints  relate  to  excellence  in  the  sense  of  a  "deluxe" 
product  with  the  extra  features  vs.  a  standard  product.  Esthetic  features, 
in  terms  of  attractiveness  are  usually  a  by-product  in  the  case  of  the  defense 
product  and  not  a  conscious  goal  in  the  contractual  sense. 

In  the  contractual  sense,  quality  is  defined  by  a  product  description 
in  a  specification ,  catalog  number  or  other  definition.  The  definition  is 
specific  and  does  not  permit  any  cnaracteristic  other  than  those  defined. 

There  are  no  variations  permitted.  The  firm  is  expected  to  deliver  exactly 
what  was  requested.  There  is  often  confusion  introduced  by  the  contractual 
inclusion  of  sampling  plans  in  a  permissive  sense.  The  contractor  may  sample 
and  the  government  will  take  a  chance  on  unknowingly  accepting  non-conforming 
material.  However,  no  non-conforming  material  is  permitted  to  be  knowingly 
accepted.  In  other  words,  the  DoD  procurement  method  may  be  defined  as  a 
"zero  defect"  but  not  a  "zero  risk"  system. 

Specificity  of  quality  definition  is  at  best  difficult,  other  than 
to  say  the  quality  requirement  is  fulfilled  when  a  product  is  delivered  that 
meets  the  requirements  specified.  In  order  to  ascertain  how  the  private 
sector  specifies  quality,  the  questionnaire  respondents  involved  in  this  study 
were  asked:  (1)  Is  it  realistic  to  specify  a  quality  level?  and  (2)  What 
term  of  specificity  is  recommended?  The  results  were: 

Question  1:  Is  is  realistic  to  specify  a  quality  level? 


Yes  No  No  Response 

83%  1 5%  2% 

Question  2:  What  term  of  specificity  is  recommended? 

1.  Performance  Specification  35% 

2.  Design  Specification  24% 

3.  Acceptable/Unacceptable  Defect 

Levels  21% 

4.  Part  Number  1% 

5.  Catalog  Description  5% 

6.  Brand  Name  4% 

?.  Market  Grade  4% 

In  the  above  questions,  respondents  responded  to  more  than  one 


answer  option.  The  percentages  shown  represent  a  weighted  finding. 
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As  the  questionnaire  results  indicate,  the  overwhelming  industry 
method  is,  as  in  government,  to  specify  precisely  what  is  desired.  Levels 
of  quality  are  not  a  rational  measure  either  in  government  or  industry  and 
the  term,  "level,"  adds  confusion  in  most  cases.  When  product  characteristics 
are  defined,  quality  is  defined. 


Government  personnel  assigned  the  responsibility  for  acceptance  of 
material  take  title  to  the  material  based  on  their  conviction  that  the  material 
meets  all  requirements.'  Despite  this  premise,  there  is  still  much  discussion 
about  lack  of  quality  and  quality  complaints. 

There  is  a  need  to  provide  a  bridge  between  the  DoD  quality  definition 
and  a  quality  product  definition.  One  aid  in  providing  this  bridge  or  gap  in 
transition  may  be  to  add  a  definition  of  a  quality  product.  A  suggested 
definition  is  the  following:  A  quality  product  is  one  that  has  a  design 
incorporating  all  of  the  users  requirements  and,  in  fact,  conforms  toaTl  design 

requirements  both  static  and  dynamic.  There  are  two  principal  advantages  to 

incorporating  this  definition  in  policy:  (1)  it  is  hardware  oriented,  not  a 
theoretical  concept,  and  (2)  it  incorporates  the  concept  of  quality  (excellence) 
of  design.  It  will  provide  recognition  that  designers  have  a  quality  responsibility 
that  must  be  discharged  if  a  quality  product  is  to  be  produced. 


In  the  interests  of  clarity  and  consistency,  the  following  terms 
and  associated  definitions  will  be  used  consistently  throughout  the  remaining 
sections  of  this  report.  Some  definitions  are  direct  quotes,  as  indicated 
by  the  reference,  others  are  direct  recommendations  from  this  research. 


(1)  Management  Quality  Characteristics  -  Those  instructions  which  relate 
to  administrative  aspects  of  Section  4  of  Specifications  or 
other  technical  product  descriptions  such  as  classification 
of  examination  and  tests,  sampling,  lot  formation  or  other 
pertinent  inspection  information  not  directly  associated 
with  a  specific  test  or  examination. 

(21  Product  Quality  Data  -  That  data  which  is  derived  from  the  inspection 
of,  and  relates  to  the  degree  of  conformance  with,  design 
requirements. 

(3)  Product  Quality  Requirements  -  Those  specific  product  tests/measurements/ 
examinations  identified  which  are  required  to  be  performed  in 
Section  4  of  the  Specification,  or  other  product  technical 
descri ptions . 


(4)  Quality  -  The  composite  of  material  attributes  including  performance 

features  and  characteristics  of  a  given  product  or  service  to 
satisfy  a  given  need.  (DoDD  4155.1) 

(5)  Quality  History  Data  -  That  government  data  which  is  assembled  and 

maintained  and  which  describes  a  firm's  performance  in  terms  of 
product  quality  data  and  the  excellence  of  quality  management 
systems . 

(6)  Quality  Management  Data  -  That  data  which  the  government  representative 

collects  and  uses  to  ascertain  the  conformance  of  the  contractors 
quality  management  system  requirements. 
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(7)  Quality  Management  Systems  -  Those  standardized  requirements  identified 

in  Section  IV  of  the  DAR  as  contract  quality  requi rements ,  e.g., 
Std  Form  32,  MIL-Q-9858 . 

(8)  Quality  Product  -  A  product  that  has  a  design  incorporating  all  of  the 

user  requirements  and,  in  fact,  conforms  to  all  design  require¬ 
ments  both  static  and  dynamic. 

(9)  Quality  Policy  -  Any  policy,  regardless  of  the  identification  of  the 

issuing  activity,  that  establishes  policy  or  results  in 
the  establishment  of  or  measurement  of  product  technical 
requirements. 

(10)  Technical  Characteristics  -  All  requirements  and  descriptions  that 

describe  the  item,  material  or  process.  Such  descriptions 
include,  for  example,  character  of  the  material,  formula, 
design,  construction,  performance,  reliability,  chemical 
composition,  electrical  and  physical  requirements,  dimensions, 
weights,  color,  name  plates,  workmanship,  standards,  etc. 

(11)  Technical  Requirement  -  Those  specific  product  requirements,  dynamic  (per¬ 

formance)  or  static  that  are  specified  in  Section  3  of  Specifi¬ 
cations  or  other  product  technical  descriptions. 

B.  ANALYSIS  OF  RELIABILITY  AND  QUALITY  POLICY 

This  section  will  examine,  in  broad  terms,  policy  in  the  general 
areas  of  reliability  and  quality.  While  the  principal  quality  related 
documents  must  be  reviewed  individually,  there  is  a  need  to  have  an  overview 
of  policy  in  order  to  have  a  sense  of  the  individual  documents  as  they 
interact  with  each  other. 

1  .  Rel iabi 1 ity 

Methodology  has  been  developed  and  is  in  use  that  permits  a 
reasonable  clear  specification  of  requirements.  The  method  of  arriving 
at  requirements  was.  di scussed  in  Section  II.  The  most  commonly  accepted 
method  of  measuring  reliability  is  contained  in  MIL-STD  781,  Reliability 
Design  Qualifications  and  Product  Acceptance  Tests:  Exponential  Distribution, 

21  October  1977.  This  is  the  C  or  third  version  of  MIL-STD  781, 

MIL-STD  781  provides  a  reasonably  rational  assessment  method  for 
determining  the  time  related  performance  of  equipment.  There  has  been  a 
wide  divergence  of  opinion  as  to  the  accuracy  of  these  projections.  In  1973 
on  one  aircraft,  the  reliability  prediction  of  the  Bomb  Navigation  System  was 
shown  to  be  higher  than  field  performance  by  a  factor  of  20  to  l.20  In 
1 975 ,  based  on  16  types  of  avionic  equipment  in  nine  types  of  aircraft 
the  disparity  was  shown  to  an  average  of  5. 88. 21  Presumably,  these  equipment 
reliability  values  were  derived  from  an  older  version  of  MIL-STD  781.  It  is 
probable  that  the  current  version  of  the  standard  will  result  in  a  better 
predi ction . 


20  Avionics  Reliability  Study,  Phase  1,  Hqtrs.  AFSC  XRX 

21  Rome  Air  Development  Center,  Operational  Influences  on  Reliability, 

RADC  TR-76-366,  December  1  976  --  •  -  , 
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Reliability  is  normally  a  relationship  between  performance  and 
time;  that  is,  performance  at  required  levels  for  some  defined  period  of 
time.  It  may,  however,  take  other  forms  such  as  in  the  case  of  a  missile 
engine  that  must  work  only  once,  i.e.,  a  one-shot  device.  The  normal 
specification  is  in  terms  of  mean  time  between  failure  or  MTBF;  this  term 
is  commonly  known  and  recognized  and  specified.  The  test  normally  consists 
of  operating  the  equipment  in  its  use  environment  for  the  period  of  time 
necessary  to  provide  some  level  of  confidence  that  the  equipment  has  met 
its  MTBF  requirement.  ■ 

Reliability  demonstrations  are  a  difficult  and  taxing  process. 

There  are  both  technical  and  administrative*  problems  to  be  overcome. 

Technical  problems  of  demonstrating  reliability  will  not  be  discussed. 

One  category  of  administrative  problems  that  arises,  results  from  the 
difficulty  of  establishing  a  clear  cut  refereeing  or  decision  making 
process  that  will  result  in  quick  and  proper  determination  as  to  whether 
a  failure  will  be  counted  or  not.  Failure  must  be  categorized  promptly 
and  accurately,  and  determinations  must  be  made  regarding  corrective  action. 
Improper  management  here  will  certainly  lead  to  inaccurate  predictions 
negating  the  benefits  of  sound  statistical  planning.  Test  results  may 
result  in  the  delay  of  a  program  at  a  crucial  time  if  they- indicate 
non-conformance  with  requirements. 

When  a  failure  occurs  that  threatens  the  schedule  or  indicates 
that  the  required  performance  is  not  available,  the  program  manager  is 
subject  to  severe  pressure  from  his  management  and  from  the  contractor 
and  even  from  his  staff. 

Contractor  pressures  obviously  result  from  a  desire  to  keep  the 
program  progressing  on  schedule.  Program  manager  assignments  are  commonly 
perceived  as  a  career  enhancement  assignment,  particularly  when  the  program 
is  concluded  meeting  all  requirements  of  performance,  schedule  and  cost. 
Reliability  testing  results  are  highly  visible  and  a  failure  which  may  lead 
to  redesign  can  cause  a  significant  delay  with  all  of  the  attendant  program 
complications  and  possibly  a  direct  impact  on  costs  and  schedule.  The 
Defense  Science  Board  has  stated  that  ..."specialists  in  the  'ilities' 
operate  under  a  professional  set  of  values  and  objectives  which  are  largely 
independent  of,  and  not  obviously  directly  supportive  of  the  goals  of  program 
timeliness  and  achievement  of  established  acquisition  schedule."  22 

The  program  manager,  although  highly  motivated  to  have  a  success¬ 
ful  program,  is  faced  with  a  wide  variety  of  major  participants  whose  motivations 
may  be  opposed  to  each  other  and  some  times  to  him.  Some  of  these  participants 
include  contractor  personnel  with  incentive  provisions  in  a  contract,  and  staff 
reliability  experts  who  expect  the  equipment  to  meet  performance,  e.g., 
reliability,  requirements  established  in  consonance  with  the  user  and  his 
operational  needs.  It  is  difficult  to  find  flaws  in  reliability  policy.  It 
is  possible  though  that  unrealistic  expectations  resulting  from  user  needs 
that  may  extend  the  limits  of  achievable  technology  coupled  with  optimistic 
performance  promises  from  industry  can  make  the  policy  unworkable  in  practice. 


22  Defense  Science  Board,  Office  of  the  Under  Secretary  of  Defense  for  Research 
and  Engineering,  Report  of  the  Acquisition  Cycle  Task  Force.  Washington,  D  C 
15  March  1978  “  “  ’  ' 


54 


2 .  Qua  1 i ty 


As  in  the  case  of  reliability,  quality  methodology  has  been  developed . 
Unlike  reliability,  however,  there  is  no  universally  agreed  upon  measure  of 
quality  such  as  .mean  time  between  failure.  At  this  stage  of  organizational 
development,  reliability  and  quality  are  considered  to  be  so  different  that  they 
are  distinctly  separate  organizations  and  the  reliability  dimension  of  product 
quality  is  likely  not  to  be  a  quality  consideration.23 

MIL-STD  961  contains  instructions  for  the  preparation  of 
specifications.  The  primary  elements  of  specifications  are  Section  3 
which  provides  a  technical  description  of  the  product  and  Section  4,  the 
quality  assurance  section,  which  provides  inspection  instructions.  Through 
the  technique  of  tiering  or  incorporating  other  specifications,  e.g., 

AIR-STD  12/19(2),  Electro  Magnetic  Compatibility  Test  Methods  for  Aircraft 
and  Electrical  Equipment,  instructions  detailing  test  procedures  are 
incorporated,  or  sometimes  specific  instructions  are  contained  in  the 
contract  proper, 

Quality  is  normally  specified  in  terms  of  material  characteristics 
dimensions  and  instant  performance  parameters,  e.g.,  shaft-  speed  1600  RPM. 

If  it  is  further  specified  that  the  device  must  operate  at  1600  RPM  for 
10,000  hours  with  no  more  than  10  interruptions  (or  failures),  then  a 
reliability  factor  is  added,  i.e.,  an  MTBF  of  1,000  hours.  The  instructions 
that  are  contained  in  MIL-STD  961  constitute  the  principal  documentation 
devoted  to  how  to  specify  quality  characteristics. 

Product  quality  requirements  are  normally  specified  by  product 
designers  as  a  routine  task  compared  to  the  creative  aspects  of  design 
concepts.  Designs  have  a  characteristic  of  quality  in  the  sense  of  their 
inherent  capability  to  provide  the  performance  which  is  required.  Design 
technical  characteristics  in  military  applications  are  usually  limited  to 
the  functional,  as  opposed  to  esthetic,  parameters.  This  is  as  opposed  to 
commercial  products  where  appearance  has  an  important  impact  on  sales  and 
subsequently  on  the  firm's  success  or  failure.  The  designer  is  responsible 
for  the  quality  of  the  product’s  design.  If  the  user  needs  are  not  clearly 
specified,  the  product's  ability  to  perform  the  intended  function  will  be  a 
matter  of .chance*  The  degree  of  conformance  to  a  misstated  design  require¬ 
ment  is  meaningless  and  suitability  or  fitness  for  use  will  not  be  achieved. 

The  centra  thrust  of  the  DoD  quality  assurance  policy  is  the 
contractor  responsibility  concept.  This  concept,  basically  stated,  is  that 
the  contractor  is  responsible  for  producing  a  product  that  meets  all  require¬ 
ments  and  proving  that  it  does  before  it  is  delivered  to  the  government.  This 
concept  is  reinforced  with  numerous  standard  contractual  devices  to  assure 
that  the  contractor  is,  in  fact,  certain  that  the  material  does  conform  to 
the  requirements  before  he  provides  it  for  acceptance. 

These  devices  fall  into  two  categories:  (1)  a  group  of  specific 
clauses  for  inclusion  in  the  contract  that  normally  incorporates  penalty 
provisions  in  the  event  of  defective  material,  and  (2)  a  family  of  management 


23  Weiss,  Bernard  L.,  Observations  and  Recommendations  to  Enhance  Product 
Quality  in  the  Operational  Environment,  (Quality  Horizons)  Air  Force 
Systems  Command,  1979 
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systems  that  require  the  contractor  to  implement  certain  quality  manage¬ 
ment  related  techniques.  These  management  systems,  their  selection  for 
inclusion  in  contracts  and  their  enforcement  have  come  to  be  a  primary 
consideration  for  quality  assurance  personnel.  The  management  system  most 
commonly  incorporated  in  AFSC  system  contracts  is  MIL-Q  9858A,  Quality  Program, 
and  it  is  commonly  reinforced  with  MIL-STD  1520A,  Corrective  Action  and 
Disposition  System  and  MIL-STD  1535A,  Supplier  Quality  Assurance  Program 
Requirements. 

Quality,  in  light  of  the  observations  above,  is  comprised  of 
three  elements— quality  of  design,  quality  of  conformance  and  quality  of 
the  contractor's  quality  management  system/  An  illustration  of  these 
elements,  or  the  quality  triad,  is  shown  in  Figure  IV-1 . 

Figure  IV-1 

Elements  of  Quality:  The  Quality  Triad 


The  three  elements  of  quality  (quality  of  design,  quality  of 
conformance,  and  quality  of  the  contractor's  management  system)  are  all 
measured  or  monitored  in  various  ways  and  by  differing  organizations. 
Quality  of  design  is  primarily  the  responsi bil ity  of  the  engineering 
element.  The  quality  of  the  product  design  is  vital  to  the  development 
of  a  successful  system.  Ooviously,  a  poor  design  will  produce  a  system 
below  the  potential  it  would  have  with  an  improved  design.  The  design 
review  process  is  utilized  by  many  contractors,  however,  there  is  little 
uniformity  in  who  attends  the  reviews.24  USAF  policy  guidelines  directs 
quality  assurance  personnel  to  participate.  However,  the  instructions  are 
vague  or  the  emphasis  is  directed  toward  produceabil ity  and  testability 
aspects  as  opposed  to  quality  of  the  design.25 


24  INTEC  survey  questions  3  and  4,  see  Appendix  A. 

25  AFR  74-1  Paragraph  3a,  3c,  AFSCR  74-1  Paragraphic  a  (1). 
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The  defined  duties  of  quality  assurance  engineers  are  to  inter¬ 
face  with  design  engineers  and  not  to  consider  the  quality  of  design  but 
rather  "assure  that  the  design(s)  consider  quality  assurance  factors,  e.g., 
process  constraints,  testing  limitations,  and  tooling  concepts  as  a  medium 
of  inspection.1,  6  Quality  of  design  is  not  identified  as  an  element  of  the 
quality  engineer's  role. 


Quality  of  conformance,  the  second  element  of  the  Quality  Triad, 
has  been  and  remains  the  traditional  role  of  quality  assurance  organizations. 
While  that  role  has  remained  unchanged,  there  have  been  major  changes  in  the 
assessment  of  product  quality  which  has  undergone  drastic  changes  as  a  result 
of  the  establishment  of  the  contractor  responsi bil i ty  concept  and  the  earlier 
development  of  sampling  as  a  government  quality  assessment  technique.  Pol i c i 
and  procedures  for  the  USAF  in-plant  quality  assurance  program  are  contained 
in  AFCMDR  74-1,  Procurement  Quality  Assurance  Program.  There  currently  is  in 
preparation  the  AFSC  Quality  Assurance  Handbook  which  provides  further  policy 
and  guidance,  and  relates  primarily  to  the  quality  assurance  function  in  the 
major  system  acquisition  process. 

AFCMDR  74-1  establishes  the  procedure  by  which  the  USAF  inspector 
accepts  or  rejects  the  material  offered  by  the  contractor- and  it  meSrswie.: 
the  quality  of  the  contractor's  quality  management  system  simultaneously. 

The  acceptance  of  products,  which  presumes  conformance  to  design  requirements 
is  predicated  on  confidence  that  the  contractor's  quality  policies  and  pro¬ 
cedures  conform  to  the  terms  and  conditions  of  the  contract  and  are  complied 
with  by  employees.27  A  variety  of  techniques  for  providing  this  confidence 
are  utilized  including  procedure  reviews,  assurance  of  conformance  to  the 
procedures,  verification  of  the  accuracy  of  the  cojatP&qtor ' s  inspection 
decision  and  independent  inspection  by  the  USAF  inspector^  This  procedure 
was  originally  developed  in  the  late  1 950 ' s  and  hartrergn  utilized  since  then 
with  continuing  refinements. 


es 


The  quality  of  the  contractor's  quality  management  system,  e.g., 
quality  program,  is  the  third  element  of  the  Quality  Triad.  The  continuing 
reduction  in  quality  personnel  has  led  to  an  increasing  reliance  on  the 
concept  that  a  good  quality  program  will  produce  conforming  material.  This 
increasing  emphasis  has  led  to  a  concentration  of  emphasis  on  the  contractor 
management  system.  A  recent  series  of  discussions  with  quality  personnel  in 
several  project  offices  in  the  Washington,  D.C.  area  indicated  that  some 
personnel  had  significant  misunderstanding  concerning  the  relationship  between 
product  qua! ity  and  management  system  quality.28 

Quality  of  design  is  not  perceived  to  be  a  quality  function. 

Policies  relative  to  design  quality  are  in  broad  sweeping  concepts  that  relate 
to  overall  system  performance  including  life  cycle  cost.  Policy  relating  to 
design  quality  was  not  located  in  this  investigation  and  quality  assurance 
policy  for  quality  engineers  is  concerned  with  manufacturi ng/testi ng  factors. 
In  short,  quality  policy  does  not  relate  to  design  quality. 


26  AFSCR  74-1,  Paragraph  2  h 

27  AFCMDR  74-1,  Paragraph  5-11,  3  January  1977 

28  Interview  with  personnel  from  the  Defense  Material  and  Standardization 
Office,  24  April  1980 
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Quality  of  conformance  is  determined  by  a  long-standing  in-plant 
program.  The  AFCMD. approach  is  fundamentally  based  on  evaluation  of  pro¬ 
duction  processes  with  limited  product  inspection  performed  to  verify 
evaluation  findings.  Recent  policy  revisions  require  specific  Q&A  assess¬ 
ments  concerned  with  both  product  quality  and  quality  planning.  Such  terms 
as  quality  levels,  quality  characteristics,  quality  program  requirements  and 
quality  planning  are  frequently  used,  confusing  the  distinct  issues  of  quality 
of  product  versus  qual.ity  of  management. 

Quality  of  the  contractor's  management  system  is  the  dominant 
feature  of  quality  policy.  Its  selection  for  contract  inclusion,  and  its 
management  by  the  government  is  thoroughl y' documented .  Quality  of  management 
systems  appears  to  be  almost  synonymous  with  product  quality.  The  current 
program  is  a  refinement  of  a  program  implemented  20  years  ago.  Insofar  as 
assessing  the  quality  of  the  contractor  management  is  concerned,  no  significant 
fault  can  be  found  with  the  program. 

C.  ANALYSIS  OF  QUALITY  RELATED  DOCUMENTS 

1  .  0MB  Circular  A-109 

This  circular  was  prepared  as  a  result  of  Executive  Branch  and 
Congressional  concerns  over  the  effectiveness  of  the  management  of  major 
system  acquisitions.  It  is  very  broad  in  nature  consisting  of  a  flexible 
philosophy  of  system  acquisition.  As  expected  in  a  directive  emanating  from 
this  level,  guidance  is  broad  in  concept,  it  does  deal,  however,  with 
management  objectives.  The  first  objective  is  that  the  system  should  operate 
effectively  in  its  intended  environment  and  demonstrate  a  level  of  performance 
and  reliability  that  justifies  its  cost.  A  second  objective  is  to  "provide 
strong  checks  and  balances  by  ensuring  adequate  test  and  evaluation."  A-109, 
although  it  does  not  use  the  word  quality,  is  obviously  concerned  with  the 
development  of  a  system  that  is  demonstratively  a  quality  product.  Performance 
and  reliability  are  clearly  subsets  of  overall  product  quality. 

2-  Department  of  Defense  Directives  5000.1  and  5000,2 

DoDD  5000.1,  Major  Systems  Acquisition,  establishes  the  basic  DoD 
system  acquisition  policy.  It  only  mentions  quality  indirectly  in  the 
Standardization  and  Operability  Section  where  logistic  supportabi 1 i ty  is  clas¬ 
sified  as  a  design  responsi bi 1 ity  equatable  to  cost,  schedule  and  performance. 

^  DoDD  5000.2,  Major  System  Acquisition  Procedures,  provides  policy, 
instructions  and  procedures  for  major  systems.  Paragraph  E.,  on  acquisition 
planning  identifies  quality  as  one  of  several  considerations  of  standardization 
of  design.  There  is  also  a  requirement  for  the  inclusion  of  MIL-Q-9858A  as 
a  quality  management  system  in  contracts.  There  is,  in  contrast,  in  the  same 
section,  an  entire  paragraph  of  equipment-based  requirements  for  the  inclusion 
of  reliability  and  maintainability  goals.  There  is,  by  comparison,  no  mention 
of  inclusion  of  a  reliability  or  maintainability  management  system. 
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The  Decision  Coordinating  Paper  (DCP)  format  includes  a  section, 
Enclosure  4,  devoted  to  performance  which  includes  gross  overall  performance 
factors  and  time  based,  i.e.,  reliability  and  maintainability  performance.  The 
Integrated  Program  Summary  (IPS)  format  contains  parts  that  relate  to  quality 
directly  or  indirectly.  Paragraph  13,  Test  and  Evaluation,  requires  test 
objectives  and  test  results.  Paragraph  16,  Reliability  and  Maintainability, 
requires  identification  of  R&M  parameters,  achievements  and  goals  for  the 
next  phase.  This  requirement  is  broken  out  on  a  milestone  basis,  i.e., 
Milestones  I,  II  and  III,  and  shows  an  overall  picture  of  reliability  and 
maintainability  growth  of  the  system.  Government  Furnished  Equipment  is 
included  in  this  analysis.  Paragraph  17  incorporates  the  Independent 
Quality  Assessment  requirements  in  DoDD  41-55.1 

These  documents  are  representive  of  current  policy  approaches  which 
concentrate  on  the  reliability  parameter  of  overall  system  quality.  The  almost 
total  reliance  on  reliability  data  for  performance  data  is  limiting  and  is 
biased  toward  answering  logistics  costs  questions,  whereas  the  first  problem 
is  a  system  that  meets  all  user  requirements.  Failure  to  meet  the  requirements 
are  most  often  identifiable  early  in  full-scale  development  cycle.  However, 
the  official  (i.e.,  contract  specified  performance  goal)  identification  of  a 
system/subsystem  problem  at  the  time  of  reliability  demonstration  test  is 
neccessarily  late  in  the  development  cycle.  DoDD  5000.2  could  be  improved 
from  a  product  quality  standpoint  by  the  addition  of  a  separate  annex  which 
describes  product  Quality  of  subsystem  (or  lower  level)  components  in  terms 
of  performance  goals  which  are  contractual ly  specified. 

3 .  Department  of  Defense  Directive  4155.1 

DoDD  4155.1  requires  each  of  the  DoD  components  to  establish 
and  manage  a  "quality  program"  which  fundamentally  requires  a  cost  effective 
management  program  to  assure  that  material  purchased  conforms  to  requirements. 

It  further  establishes  the  basic  contract  administration  relationship  between 
contractor  and  government  quality  assurance  personnel,  commonly  called  the 
contractor  responsibily  concept.  It  is  primarily  and  properly  a  management 
oriented  document  with  limited  product  related  instructions  as  compared  to 
DoDD  5000.2. 

Since  .this  directive  must  encompass  all  material  purchased  by  the 
DoD,  it  cannot  be  directed  exclusively  toward  the  quality  of  major  systems. 

It  does,  however,  include  requirements  for  review  of  the  quality  of  such 
material.  In  view  of  the  intent  of  this  study,  the  following  five  policies 
v paraphrased)  of  particular  significance  are  identified  for  later  consideration! 

(a)  Warranty  clauses  shall  be  used  as  a  device  to 
hold  contractors  responsible  for  their  products. 

(b)  The  DoD  components  shall  consider  using  incentive 
fee  arrangements  for  achieving  quality  goals. 

(c)  DoD  components  shall  assure  that  contractors  with 
a  record  of  providing  unsatisfactory  products  do 
not  receive  more  contracts. 
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(d)  DoD  components  and  system  managers  shall  assure 
the  adequacy  of  quality  requirements  at  DSARC 
decision  points  (milestones). 

(e)  A  product  deficiency  reporting  and  data  feedback 
system  shall  be  established. 

There  are  certainly  other  elements  of  DoDD  4155.1.  However,  these 
are  principal  policies  that  relate  to  the  essence  of  this  study. 

A.  The  Defense  Acquisition  Regulation  and  Quality  Policy 

a .  General 

The  DAR  is  a  procurement  regulation  intended  to  establish 
uniform  procurement  policies.  Procurement  instructions  in  it  deal  primarily 
with  the  purchasing  function  in  the  areas  of  contract  award  and  contract 
administration.  It  also  is  concerned  with  the  interface  of  all  of  the 
personnel  who  have  an  interest  in  products  under  contract  and  in  some  cases 
defines  responsibility  of  such  personnel.  In  the  interests  of  uniform 
procedures,  it  has  become  necessary  to  standardize  certain  functions  such 
as  the  use  o^  inspection  stamps  and  the  control  of  government  property. 

b.  Section  XIV,  Procurement  Quality  Assurance 

The  DAR  recognition  of  the  acceptance  function  with  its 
associated  elements  of  assuring  that  the  product  conforms  to  established 
requirements  is  evidenced  by  the  existence  of  an  entire  section  devoted  to 
quality  assurance.  The  majority  of  Section  XIV  is  devoted  to  inter-government 
quality  assurance  organizational  roles  and  responsibilities.  Section  XIV 
recognizes  three  government  elements:  the  organization  responsible  for 
technical  requirements,  the  purchasing  office,  and  the  contract  administration 
office.  Product  quality  considerations  are  minimal  and  restricted  to  areas 
of  the  contract-administration  office  duties.  The  technical  office  is 
responsible  for  prescribing  contract  quality  requirements  such  as  inspection 
and  testing  and  are  permitted  in  special  cases  to  issue  instructions  concerning 
necessary  government  inspection.  The  purchasing  office  is  responsible  for 
contractua  1  i  zi  ng'/formal  i zi  ng  requirements  for  quality  and  issuing  government 
inspection  instructions  to  the  contract  administration  activity  and  for 
specifying  a  quality  management  program  for  the  contractor. 

Section  IV  provides  extensive  guidance  on  the  contract 
administration  element  of  quality  management.  There  is  no  guidance  in 
Section  XIV  that  furthers  the  DoDD  4155.1  instructions  pertaining  to 
warranties,  incentive  fees,  major  systems,  or  motivational  considerations 
or  policy. 


The  section  in  Paragraph  14-404  does  require  that  selected 
government  records  be  established  which  document  acceptability  of  products 
and  processes  that  could  contribute  to  the  requirement  noted  above,  3(C) 
for  eliminating  contractors  with  poor  product  quality  records.  There  is  no 
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indication  that  such  information  is  to  be  continually  updated  and  provided 
to  a  central  data  source  or  to  be  considered  as  an  element  of  preaward 
surveys  for  prospective  contractors.  There  is  no  reference  to  other  parts 
of  the  DAR  where  selection  of  contractor  or  determination  of  profit  is 
discussed  in  a  -motivational  concept,  i.e.,  Section  3. 

There  is  also  a  requirement  for  the  collection  of  "quality 
data,"  14-405,  reported  by  users  during  "initial  use"  which  could  contribute 
to  both  the  3(c)  and  3'(e)  (data  feedback)  requirements  listed  above.  The  Quality 
Horizons  Study  noted  a  continuing  need  for  data  feedback  and  its  ultimate 
use  in  a  comprehensive  business  strategy  concept  involving  contractor 
motivational  related  approaches  such  as  publicity  and  reduced  surveillance. 

c .  Sections  I  and  III 

These  sections  of  the  DAR  are  discussed  together  since  the 
material  they  contain  is  interrelated  and  associated  generally  with  the 
prime  procurement  functions  of  preparing,  negotiating  and  administrating 
contracts.  Product  quality  is  mentioned  frequently  in  these  sections  as 
an  element  of  deliberation  concerning  the  prime  functions.  It  is  here  also 
that  motivation  appears  as  a  government  issue  and  policy.-  And  it  is  here 
and  Section  XIV  that  jointly  affect  the  manner  in  which  the  USAF  policy 
is  formulated  effecting  quality  administration  and  product  quality. 

(1 )  Section  I 

Part  III  of  Section  I  establishes  the  general  policies 
to  be  followed  in  Defense  contracting,  the  first  of  which  is  that  procure¬ 
ments  shall  be  made  on  a  competitive  basis  to  the  maximum  possible  extent. 

The  section  also  establishes  the  time  of  delivery  as  an  essential  element 
of  a  contract  (1-305).  Section  I  also  contains  the  Liquidated  Damages  clause 
(1-305),  a  penalty  oriented  mode  of  motivation.  Paragraph  1-313  discusses 
the  procurement  of  parts,  i.e.,  spare  parts.  Quality  is  a  consideration 
in  that  certain  non-standard  parts  may  be  purchased  from  firms  that  have 
not  provided  them  before  only  if  adequate  data  including  test  results 
and  quality  assurance  procedures  are  available  for  government  usaqe.  (See 
DoDD  4155.1,  paragraph  C.4.G.) 

1 

Product  quality  informtion  appears  as  a  factor  in 
renegotiation  performance  reports  (1-319  d)  (viii)  (A).  Product  quality 
is  a  consideration  in  multi-year  procurement  policy  guidance.  Paragraph 
1-322  (b)  (3)  (ii).  Policy  for  warranties  is  established  in  1-324,  where 
contractor  quality  management  system  is  a  factor  in  that  MIL-Q-9858  and 
MI L- I -45208  are  considerations  in  the  application  of  a  warranty.  In 
paragraph  1-326  the  component  breakout  program  is  discussed.  Quality, 

Reliability  and  Performance  must  be  evaluated  prior  to  a  breakout  decision. 
Paragraph  1-406  identifies  the  functions  that  are  normally  performed  by 
contract  administration  organizations  including  Procurement  Quality 
assurance.  In  paragraph  1-900  responsible  prospective  contractors  are 
discussed.  Recent  unsatisfactory  performance  in  quality  and  delivery  are 
identified  in  Paragraph  1-902  as  examples  of  what  must  be  resolved  prior 
to  making  an  affirmative  decision  of  contractor  responsibility.  Contractor 
quality  management  system  inadequacies  are  also  a  listed  consideration. 
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The  Qualified  Products  List  is  discussed  in  Paragraph  1-1110.  A 
history  of  delivery  of  products  not  meeting  requirements  is  to  be 
provided  to  the  specification  preparing  activity.  In  1-1803  past 
performance  of  contractors  is  identified  as  one  consideration  when 
determining  the  need  for  post-award  orientation  of  contractors. 


Part  1-2100  Procurement  Planning  requires  a  formalized, 
structured  procurement  plan  for  major  development  and  production  contracts. 

The  program  manager  is  expected  to  consult  with  various  management  disciplines 
such  as  engineering,  production,  quality  assurance,  etc.  in  preparing  the 
plan.  A  sample  format  is  included  illustrating  the  elements  of  the  plan 
and  including  such  areas  as  Government  Furnished  Material ,  R  &  M,  Life  Cycle 
Cost  (LCC),  Should  Cost  and  Test  and  Evaluation  Approach,  etc.  However, 
quality  planning  is  not  included  as  a  separate  element. 


Section  I  provides  considerable  policy  guidance  for 
procurement.  It  identified  in  numerous  paragraphs,  where  the  contractor 
product  quality  history  is  to  be  a  consideration  in  procurement  decisions. 
The  contractor  product  quality  data  requirements  in  DoDD  4155.1  and 
Section  XIV  clearly  have  application,  if  available,  to  contracting  officers. 
The  procurement  plan  of  1-2100  discusses  numerous  contract  considerations. 
Product  quality  information  is  fractionated  into  several  areas  with  no 
clear  cut  requirement  for  it  or  for  quality  planning  as  a  primary  concern. 
This  appears  to  be  a  significant  omission. 


(2)  Section  III 


Section  III  is  concerned  with  negotiated  procurement. 
The  negotiated  procurement  normally  offers  opportunity  for  consideration 
of  the  merit  of  the  firm's  Quality  Management  System  and  the  utilizatio- 
of  previously  generated  quality  history  data.  An  early  opportunity  for 
the  utilization  of  quality  history  data  may  occur  when  it  becomes  neces 
for  the  preparation  of  a  Determination  and  Findings  (D&F) .  (3-301) 


sary 


There  are  numerous  contract  types  identified  in  Section 
III  including  principally,  fixed  price  and  cost  reimbursement.  Either  of 
these  types  may  include  some  type  of  incentive.  Incentives  may  be  based  on 
a  variety  of  contract  objectives  depending  upon  the  issues  surrounding  a 
particular  procurement.  Examples  include  cost  incentive,  equipment  per¬ 
formance  incentive,  or  in  the  case  of  objectives  which  are  less  clearly 
definable,  an  award  fee  approach  may  be  utilized.  Incentives  may  add  to 
profit  or  take  away  from  profit  or  may  be  limited  to  either  case. 


Section  III  is  where  the  principles  of  contractor 
motivation  are  identified  and  discussed  and  where  other  contracting 
principles  are  found  which  may  unintentionally  impinge  on  contract 
objectives.  It  is  in  Section  III  that  the  principle  of  profit  is 
identified  as  the  basic  motive  of  contractors.  Contracting  personnel 
are  advised  that  the  contract  type  selected  and  negotiations  surrounding 
it  should  "ensure  that  outstandingly  effective  and  economical  performance 
is  met  by  high  profits,  mediocre  performance  by  mediocre  profits  and  poor 
performance  by  low  profits  or  losses."  It  is  expected  by  DAR  preparing 
officials  that  the  application  of  these  objectives  on  a  contract  by  contract 
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basis  will  normally  result  in  a  range  of  profits.  It  is  recognized  also 
that  there  may  be  a  time  in  the  contract  life  that  the  profit  motive  is 
secondary  and  that  technical  and  cost  risks  may  reduce  the  effectiveness 
of  profit  as  a  motivator.  In  the  final  analysis,  however,  the  contract 
type  selected  should,  and  its  provisions  should,  tie  profits  to  the 
contractor's  efficiency  in  controlling  costs  and  meeting  desired  standards 
of  "performance  reliability,  quality  and  delivery, "  (3-401).  In  other  words, 
qua! i ty/rel iabil ity  should  be  a  major  consideration  in  contract  formulation. 

The  Cost  Plus  Award  Fee  (CPAF)  (3-405.5)  contract  differs  from 
the  usual  incentive  type  of  contract  in  that  it  provides  for  awarding  an 
incentive  fee  in  contract  areas  where  the  degree  of  achieving  or  not 
achieving  success  is  not  susceptible  to  finite  determination.  The  CPAF 
fee  is  positive  in  that  it  does  not  provide  for  reduction.  It  is  composed 
of  two  parts,  a  fixed  amount  that  does  not  vary  with  performance  and  a 
separate  award  fee  that  is  believed  will  motivate  contractor  personnel  to 
improve  on  what  may  be  their  normal  performance.  ,  The  CPAF  concept  is 
applicable  to  services  where  feasibility  is  known  but  degree  of  success 
must  be  based  on  a  subjective  evaluation  of  the  services  performed.  This 
element  of  the  CPAF  concept  makes  it  applicable  to  contractor  management 
systems  such  as  those  imposed  in  the  area  of  quality  assurance. 

3-405. 5(h)  states  that  "in  certain  cases,  it  may  be  desirable  to 
motivate  and  reward  a  contractor  for  management  performance  over  and 
above  that  which  can  be  objectively  measured  and  incentivised  under 
other  forms  of  government  contracts." 

The  CPAE,  or  award  fee  provisions  should  be  evaluated  to  determine 
performance  at  stated  intervals  during  the  contract  performance  and 
contractor  personnel  should  be  made  aware  of  the  findings.  If  partial 
fee  payments  are  made  at  each  evaluation  period,  it  is  expected  that  the 
profit  motive  will  induce  the  contractor  to  improve  poor  performance  or 
continue  good  performance.  It  is  also  suggested  in  3-405. 5(e)  that 
consideration  should  be  given  to  constituting  a  board  to  evaluate  performance 
and  to  afford  contractor  personnel  the  opportunity  to  present  his  own 
eval uation . 


Firms  who  have  had  experience  with  award  fees  were  asked  in 
the  questionnaire  to  indicate  the  impact  they  believe  it  would  have  on 
material  quality.  Their  responses  were  as  follows: 


Negative 

No 

SI ightly 

Very 

No 

Impact 

Impact 

Beneficial 

Beneficial 

Response 

0% 

11% 

18?» 

16% 

55% 

Forty  five  of  the  82  firms  answering  the  survey  did  not  respond, 
giving  a  measure  of  the  degree  of  usage  of  the  award  fee  in  government  con¬ 
tracts.  The  award  fee  has  an  unusual  characteristic  in  that  it  is  simultaneously 
a  financial  and  psychological  incentive.  Discussions  with  contractor  per¬ 
sonnel  indicate  a  belief  that  the  corporate  and  government  attention  drawn 
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to  the  quality  organization  involved  in  an  award  fee  process  is  in  itself 
a  positive  motivator.  The  profit  motive  is  obvious.  However,  it  may 
well  be  overshadowed  as  a  motivator  by  the  psychological  characteristics 
of  the  award  fee  process. 

When  the  82  firms  answering  the  survey  were  asked  to  rate 
special  bonuses  or  cash  awards,  and  official  recognition  in  terms  of  their 
ability  to  motivate  employees  to  strive  for  high  quality  workmanship  levels, 
their  answers  were: 


No 

Benefit 

Slight 

Benefit 

Significant 

Benefit 

Best 

Method 

No 

Response 

Official 

Recognition 

0 % 

4% 

54% 

39% 

3% 

Special  Bonus/ 
Cash  Awards 

12  % 

35% 

37% 

10% 

6% 

These  responses  clearly  show  the  perspective  of  these 
respondents  toward  the  increased  recognition  that  is  inherent  to  the  award 
fee  method  of  contracting.  Employees  in  this  sense  are  those  that  will 
necessarily  be  recognized  by  upper  level  management  as  having  a  specific 
role  in  increasing  profits  and  improving  relationships  with  the  customer. 
There  is  also  the  direct  communication  recognition  established  due  to  the 
interaction  on  a  horizontal  level  between  contract  personnel  performing 
the  award  fee  task  and  government  personnel  evaluating  it.  It  has  also 
been  found  that  while  the  award  fee  contract  may  be  more  complex  to 
administer,  it  does  require  a  clear  determination  of  requirements  and 
identification  of  areas  of  emphasis  and  a  corresponding  understanding 
of  what  is  important  to  the  government.  Research  findings  of  other 
investigators  also  confirm  that  the  profit  motivation  is  not  the  only 
benefit  resulting  from  the  use  of  the  award  fee.  Professors  Hunt  and 
Conjeski  have  stated,  "It  also  serves  a  broader  program/project  control 
function,  stimulating  and  structuring  a  steady  flow  of  information  across 
organizational  boundaries."  26 

Most  investigators  have  found  and  discussed  positive 
aspects  of  CPAF  contracts.  These  findings,  however,  are  not  exclusively 
positive.  The  need  for  a  structured  governmental  management  effort  to 
assess  performance  and  decide  on  fees  to  be  awarded  could  obviously  have 
a  debilitating  effect  on  manpower  requirements.  Potential  problems 
encountered  on  one  major  system  were  described  by  one  author  as  follows:  27 

o  Administrative  expense  (time,  men  and  material) 
to  evaluate  the  firm's  performance  and  make  a  fee 
determination,  i.e.,  difficulty  in  convening  the 
fee  evaluation  board  and  lack  of  time  of  board 
members  for  evaluation. 


2f  Hunt,  Raymond  G.  and  Raymond  T.  Conjeski,  The  Award  Fee  Method  of  System 
Acquisition,  State  University  of  New  York,  Buffalo  1979 

Brown,  Jerry  V.,  The  Award  Fee  Incentive:  Management  Considerations 
Regarding  its  Application  to  Research  and  Deve 1 opment  Contracts ,  Defense 

Systems  Management  College,  Fort  Belvoir,  Virginia,  November  1976 
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o  Not  funding  all  of  potential  fee;  in  effect, 
signaling  to  contractor  personnel  a  possible 
lack  of  financial  determination  on  the  part 
of  the  government. 

o  A  delay  in  paying  award  fee  having  an  adverse 
effect  on  the  contractor  in  cash  flow  planning. 

It  was  reported  by  the  Logistics  Management  Institute 
that  there  was  no  compelling  evidence  (1968)  that  cost  incentives  were 
working.  The  study  did  conclude,  though,  that  the  use  of  incentives  pro¬ 
duced  "more  thorough  government  acquisition  planning  and  more  complete  and 
precise  communication  of  procurement  objectives  to  contractors . 1,28 

Despite  the  problems  that  may  be  encountered,  the  Award 
Fee  concept  is  a  methodology  that  has  the  potential  of  a  positive  influence 
on  product  quality.  It  must  be  accepted,  in  the  lack  of  evidence  to  the 
contrary,  that  USAF  personnel  are  accepting  only  material  that  conforms  to 
requirements.  Quality  management  improvements,  however,  on  the  contractor's 
part  should  lead  to  a  reduction  in  non-conforming  material  which  is  rejected 
and  must  be  replaced,  or  defective  material  which  can  be  corrected.  These 
situations  underscore  the  contention  that  quality  and  cost  effective  product¬ 
ivity  are  opposite  sides  of  the  same  coin  and  that,  therefore,  quality 
improvements  lead  to  reduction  in  costs. 

Application  of  the  Award  Fee  concept  to  quality  management 
is  consistent  with  the  three  requirements  for  a  successful  application, 
which  are: 


o  its  measurement  is  possible  (subjective) 

o  it  is  susceptible  to  contractor  control 

o  it  is  relevant  to  contract  goals 

Product  quality  is  discussed  directly  or  indirectly  in 
several  other  perspectives  in  the  DAR,  Section  HI.  These  are  discussed 
in  the  same  sense  as  they  are  in  Section  II.  It  is  possible  that  product 
quality  considerations  are  summarily  dismissed  in  these  discussions.  This 
could  occur  as  a  result  of  product  assurance  concentration  on  the  quality 
management  system  aspects  of  contracting. 

Paragraph  3-808,  which  is  concerned  with  profit/fees  in 
cost  reimbursement  contracts  previously  provided  an  opportunity  for  practically 
mandating  the  utilization  of  quality  history  data.  When  a  contract  profit 

was  based  on  cost  analysis,  the  government  negotiator's  profit  objective 
was,  among  other  things,  to  reward  contractors  with  a  good  quality  record  and 
penalize  those  whose  performance  is  poor.  This,  in  effect,  said  that  a 
contractor's  quality  history  is  a  proper  element  of  deliberation  when 
establishing  profit  factors. 


28  Logistic  Management  Institute,  An  Examination  of  the  Foundations  of 
Incentive  Contracting,  Washington,  D.C.,  May  1968 
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In  September,  1976,  in  the  Defense  Procurement  Circular 
(DPC)  76-3,  quality  and  delivery  were  deleted  as  considerations  from  the 
weighted  guideline  method  of  determining  profit  objectives  (DAR  3-808.4). 

Other  changes  were  made  in  the  guidelines  as  well.  The  deletion  of 
performance  as  a  consideration  was  explained  as  follows: 

"Past  performance  as  a  profit  determinant  has  been 
deleted  from  the  weighted  guidelines.  No  objective  measure  of  past 
performance  exists  and  the  use  of  this  factor  has  been  erratic  and  of  little 
significance  in  arriving  at  a  negotiated  profit.  This  factor  will,  however, 
continue  to  be  used  in  the  source  selection  process."  The  guidelines  now 
consist  of  (1)  contractor  effort,  (2)  contractor  risk,  (3)  facilities 
investment  and  (4)  special  factors. 

DCP  76-3  identified  the  principal  procurement  objective 
as  "obtain  a  quality  product,  delivered  on  schedule  at  a  reasonable  price." 

The  deletion  of  performance  as  a  consideration  in  the  determination  of 
profit  appears  to  take  away  a  strong  motivation  to  deliver  conforming 
material  on  time  and  within  established  costs.  3-808.4  does  still  provide 
the  opportunity  to  introduce  "other"  considerations  under  special  factors 
as  there  appears  to  be  no  reason  why  an  assessment  of  past  performance  could 
not  be  introduced  under  the  section  if  a  consistent  methodology  were  used 
which  provided  a  measurable  past  performance  factor. 

D.  SUMMARY  AND  FINDINGS 

1 .  Mot i vation 

The  DAR  presumption  of  profit  as  the  prime  motivation  of  the  firm 
is  normally  correct  in  the  classic  buyer/seller  relationship.  As  discussed 
in  other  parts  of  this  report,  however,  on  any  particular  contract,  profit 
may  not  be  the  prime  motive  of  the  firm,  depending  on  the  life  phase  of  the 
firm  and  its  particular  goals  at  the  time  of  contract  formulation.  The  DAR 
presumption  of  profit  must  have,  as  an  element  of  it,  a  corresponding  pre¬ 
sumption  of  purchases  in  a  free  market  situation;  an  assumption  that  is 
questionable  at  best.  Also,  the  profit/quality  relationship  assumption  must 
include  the  premise  that  the  profit  will  motivate  those  individuals  who  can 
directly  influence  product  quality,  e.g.,  the  operators  as  defined  earlier. 
Figure  IV  -  2  illustrates  contracting  techniques  vs  contractor  personnel. 

The  DAR  is  essentially  silent  on  motivational  guidance/management 
in  contrast  to  other  management  systems  and  programs.  The  DAR  contract 
situation  does  not  transmit  to  contractor  employees  the  sense  of  urgency 
and  participation  that  is  present  in  commercial  air  carrier  programs  where 
there  is  immediate  feedback  of  problems  when  quality  problems  impact  commercial 
transport  systems.  The  DAR  provides  no  significant  encouragement  or  direction 
for  the  firm  to  develop  motivational  programs  that  will  permit  the  operator 
to  have  a  personal  sense  of  satisfaction  or  failure  depending  on  the  success 
or  failure  of  the  USAF  systems  in  the  field. 

Despite  the  reaffirmation  that  profit  is  to  be  used  to  motivate 
the  firm,  aside  from  its  effectiveness  or  non-effectiveness,  product  quality 
history  of  previous  contractors  is  a  minor  consideration  in  determining 
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profits  on  cost  reimbursement  contracts  and  is  of  limited  importance  in 
the  source  selection  process.  Even  if  product  quality  history  was  a  major 
consideration,  it  is  unlikely,  under  the  present  methods,  that  it  would 
be  available  in  any  cohesive  usable  form  due  to  the  current  lack  of  a 
system  for  constructing  and  maintaining  the  firm's  quality  history. 

The  DAR  principally  is  concerned  with  the  instant  contract  at  any 
one  point  in  time.  There  is  no  consideration  of  developing  a  government 
strategy  that  will  understand  the  motives  and  concerns  of  the  firm.  The 
firm,  by  comparison,  may  have  a  predetermined  corporate  strategy  that  will 
color  its  negotiating  approach.  As  a  minimum,  its  approach  will  be  affected 
by  the  firm's  pre-established  hierarchy  of 'needs.  The  DAR  standard  contract 
approach  clearly  is  at  a  disadvantage  in  negotiations  with  a  firm  which  by 
contrast,  is  following  a  predetermined  corporate  strategy. 

A  collateral  profit  question  is  how  does  simple  profit  rank  as  a 
reward  on  contracts  where  there  is  reimbursement  for  an  overhead  which  may 
result  in  maintaining  hundreds  of  employees  in  a  reimbursable  overhead 
situation.  This  question  is  not  answerable  in  this  report.  However,  it 
is  worthy  of  serious  investigation.  In  some  cases,  particularly  very  large 
firms,  profit  may  be  largely  inconsequential  when  compared  to  overhead 
related  charges. 

2 .  Qual ity 

Product  quality  is  a  function  of  the  quality  of  design  and  the 
degree  of  conformance  of  the  product  to  the  design.  Quality  of  the  product 
design  is  first  dependent  upon  an  accurate  definition  of  and  translation 
from  the  user  needs  into  product  requirements.  The  second  part  of  the 
product  design  that  must  be  considered  is  its  excellence  in  terms  of  the 
ability  to  perform  the  necessary  functions  for  the  necessary  duration  of  the 
product's  life.  These  first  two  conditions  of  design  must  be  achieved 
since  the  conformance  of  product  to  a  poor  design  has  limited  benefits. 

Product  quality  is  an  all  encompassing  term.  There  is  no  logical 
division  or  distinction  between  static  characteristics,  e.g.,  material  or 
dimensional,  instant  performance  testing  and  time  dimensional  testing,  i.e  , 
reliability. 

Quality  pol icy  is  oriented  toward  the  conformance  element  of 
quality.  It  is  comprehensive  in  organizational  management  aspects  and 
concentrates  on  the  production  stage  of  the  life  cycle.  The  policy 
emphasis  concentrates  on  assuring  that  proper  quality  management  systems 
are  incorporated  in  contracts  and  that  the  firm  is  in  consonance  with  those 
system  requirements. 

Reliability  policy  by  contrast  is  product  centered.  Reliability 
organizational  concepts  can  accordingly  be  simpler,  and  are.  Quality 
considerations  are  in  essentially  all  contracts,  as  opposed  to  reliability 
requirements  which  are  in  relatively  few  contracts.  The  reliability/ 
management  problems  are  much  less  complex  and  management  controls/policy 
appear  to  be  more  simple  and  straightforward. 
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i he  significant  differences  lies  in  the  reliability  specialists 
involvement  in  establishing  and  influencing  the  quality  of  desian  in  thp 
early  or  pre-production  phases  of  the  life  cycle.  Quality  personnel,  by 
comparison,  examine  established  requirements  to  plan  bow  to  assure  confor¬ 
mance.  SAMSO-Std  73-2C  (ELECTRONIC  PARTS,  MATERIALS,  AND  PROCESSES  FOR 
SPACE  AND  MISSILE  APPLICATIONS;  STANDARD  CONTROL  PROGRAM  FOR)  is  the  only 
significant  example  found  where  a  quality  organization  is  involved  in 
influencing  the  quality  of  product  design. 

The  organizational  and  conceptual  barriers  between  Quality  and 
Reliability  organizations  does  not  have  a  positive  influence  in  that  it 
inhibits  communications .  It  also  effectively  eliminates  the  thought  that 
reliability  is  an  element  of  product  quality.  The  qual ity/rel iabil ity 
organizational  realignments  discussed  in  the  Quality  Horizons  study  will 
ease  organizational  constraints  if  accomplished.  However,  such  a  re-organi¬ 
zation  will  have  a  limited  impact,  and  will  not  solve  the  greater  issue  of 
the  identification  of  design  quality  as  a  responsibility  to  be  recognized 
and  a  task  to  be  systematically  accomplished. 
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V.  ANALYSIS  OF  VALIDITY  OF  QUALITY  ASSURANCE  AND  RELIABILITY  APPROACHES 


A.  GENERAL 

It  is  difficult  to  find  fault,  in  principle,  with  any  USAF  policy 
relating  to  the  management  of  the  quality  assurance  process.  Mo  evidence 
has  been  seen  that  indicates  any  substantial  amount  of  defective  material 
is  entering  the  inventory.  All  of  the  existing  policy  reviewed  has  a  positive 
efiect  on  product  quality  and  while  there  is  room  for  improvement,  none  is 
so  defective  that  it  should  be  abandoned.  Improvements  are  possible  and 
recommendations  are  made  elsewhere  in  the  report.  In  general,  the  policy  is 
to  maintain  a  quality  assurance  presence  and  influence  in  all’of  the  life 
phases  of  the  product  from  conception  to  disposal.  When  the  various  life 
cycle  phases  other  than  production  are  examined  in  detail,  however,  there 
are  few  functions  that  have  significant  influence  on  product  quality.  In 
order  to  recommend  mere  positive  identification  of  policy  to  continue  or 
discontinue,  it  is  necessary  to  examine  the  principal  functional  areas 
of  quality  assurance  as  described  in  AFR  74-1. 

B.  DESIGN  AND  DEVELOPMENT 

A  quality  role  in  the  R&D  process  is  essential  if  the  USAF  is  to 
influence  the  design  element  of  product  quality.  The  traditional  emphasis 
on  quality  planning  to  assure  that  conformance  to  requirements  can  be 
confined  is  necessary  and  should  be  continued.  The  recently  added  require¬ 
ment  for  product  assessments  at  the  end  of  the  demonstration  and  validation 
phase  and  the  full-scale  engineering  development  phase  should  be  beneficial. 

AFR  74-1,  Paragraph  3d,  provides  an  opportunity  to  introduce 
quality  or  excellence  of  design  as  a  prime  management  consideration. 

These  instructions  are  hampered  by  poor  definition,  i.e.,  quality  charac¬ 
teristics.  However,  it  identifies  a  missing  aspect  of  development  which 
is  an  assessment  of  design  quality..  If  these  instructions  result  in 
traditional  quality  assurance  assessments  of  design  only  in  terms  of 
conformance  measurements  problems,  the  opportunity  to  truly  have  an 
independent  assessment  of  design  will  not  be  fulfilled.  It  is  conceivable 
that  the  function'  insofar  as  design  assessment  is  concerned,  must  be 
performed  by  other  organizational  elements.  Who  does  it  is  immaterial  to  the 
objective.  This  requirement  should  also  assure  that  not  only  planning 
for  demonstrating  performance  is  achieved/performed  but  that  the  testing 
is  performed  and  the  degree  of  success  clearly  described. 

C.  CONTRACTING 

The  majority  of  the  policy  guidance  associated  with  contracting 
is  related  to  contractor  quality  management  systems  or  administrative 
instructions  concerning  the  management  systems  and  other  specifications 
and  standards.  These  instructions  are  necessary  but  may  be  considered 
routine  compared  to  other  instructions  relating  to  such  matters  as 
incentives  and  contractor  quality  history. 
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The  use  of  incentives  is  a  relatively  new  approach  to  quality 
in  contracting.  Product  quality  incentives  have  been  used  in  the  sense 
of  warranties  and  more  recently,  such  plans  as  the  Reliability  Improvement 
Warranty  (RIW)  and  Life  Cycle  Cost  (ICC)  incentives  have  been  used.  The 
incentive  approach  appears  to  have  merit  and  can  be  considered  in  terms  of 
either  award  fee  or  performance  incentive.  Current  instructions  could  be 
expanded  upon  by  elaborating  on  the  alternative  of  performance  versus  award 
fee  incentives.  Incentives  that  will  emphasize  field  operational  performance 
are  desirable  if  they  can  be  developed. 

The  development,  maintenance  and  utilization  of  data  about 
unsatisfactory  products  and  its  review  prior  to  contract  award  can  make  a 
valuable  contribution  to  developing  tactics  when  awarding  a  major  contract. 
The  availability  of  field  performance  data  is  a  constant  problem  which  was 
reaffirmed  in  the  Quality  Horizons  Study. 29If  field  data  is  not  provided  to 
purchasing  offices  routinely,  it  is  unlikely  that  this  requirement  to  withold 
contracts  from  firms  with  a  poor  quality  history  can  be  met. 

D.  CONTRACT  ADMINISTRATION 


The  dominant  principle  of  quality  policy  in  the  administration 
of  contracts  is  the  contractor  responsibility  concept.  This  concept  is 
coupled  with  the  DoD  requirement  for  some  type  of  quality  management  system, 
which  if  conformed  with,  provides  confidence  that  the  product  conforms  to 
requirements.  The  (JSAF  in-plant  approach  is  essentially  a  preventive  quality 
program  based  on  identification  and  monitoring  of  critical  parts  and  processes, 
the  monitoring  of  procedures,  and  an  emphasis  on  elimination  of  the  defects 
encountered.  Acceptance  of  products  is  based  on  confidence  that  contractor 
policies  and  procedures  are  adequate  and  are  complied  with  by  contractor 
employees.  The  present  in-plant  program,  or  some  variation  of  it,  has  been 
in  use  for  several  years.  There  is  no  evidence  to  dispute  its  success  in 
assuring  delivery  of  products  that  conform  to  contract  retirements. 

Contract  administration  policy  is  fundamentally  that  there  should 
be  assurance  that  products  conform  to  requirements.  The  system  for  assuring 
product  conformance  is  greatly  biased  toward  controlling  procedures  and  pro¬ 
cesses  with  limited  emphasis  on  government  product  inspection  or  test.  In 
as  much  as  R&D  is  done  in  the  plant  environment,  the  need  for  an  independent 
product  assessment  indicates  a  need  for  a  confirmation  of  the  findings  of 
in-plant  personnel.  Other  than  that  apparent  question,  the  effectiveness 
of  contract  administration  policy  does  not  seem  to  be  in  question. 


In  a  sense,  the  efficiency  of  the  current  methodology  is  under 
examination  in  light  of  such  programs  as  the  Defense  Logistics  Agency  (DLA), 
Contract  Assessment  Program  (CAP),  the  FAA  Quality  Assurance  Systems  Analysis 
Review  (QSARJ  and  the  FAA  Designated  Manufacturer  Inspection  Representative 
(DMIR).  The  Quality  Horizons  Program  recommended  consideration  of  the 
Minimum  In-Plant  Surveillance  Program  (-MLR.S  ) ,  In  the  case  of  the  CAP  program, 
the  DLA  does  not  uniformly  support  the  CAP  program  nor  does  the  FAA  use  the 
QSAR  and  DMIR  approach  on  equipment  purchased  by  the  agency.  It  does  appear, 
however,  that  an  invest! gat inn  to  determine _if  more  efficient  in-pTant  procedures 
can"be  developed  would  be  appropriate  at  this  time  I 


29  Weiss,  Bernard  L,,  Observations  and  Recommendations  to  Enhance  Product 
Quality  in  the  Operational  Environment  (Quality  Horizons),  Air  Force 
Systems  Command,  1979 
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In  accordance  with  contract  guidelines,  this  study  is  essentially 
limited  to  policy  considerations.  Without  a  serious  investigation  into 
in-plant  methodology,  it  is  obviously  impossible  to  provide  an  authoritative 
assessment  of  the  USAF  quality  assurance  program.  There  are,  however, 
alternatives.  As  one  illustration,  the  United  Kingdom  (UK)  approach 
resembles  the  FAA  QSAR  program  in  that  there  is  an  assessment  of  the  con¬ 
tractor's  program  and  the  award  of  future  contracts  is  dependent  upon  a 
satisfactory  finding.  The  UK  quality  assurance  program  results  in  a 
smaller  in-plant  quality  assurance  staff,  e.g.,  in  1978,  _14  at  Rolls-Royce, 
-,av€a&iljty  of  nearly  20,000  employees.  There  is  no  reason  to  believe  tfieTr’ 
aircraft  engines  are  of  poorer  quality  than,  say,  Pratt  and  Whitney  engines. 
While  there  is  no  hard  data  to  support  the  conclusion,  an  intuitive  assessment 
is  that  the  USAF  quality  assessment  program  imposes  management  controls 
and  enforces  them  to  such  a  degree  that  the  contractor  responsibility  concept 
is  a  concept  that  is  mostly  theory. 


In  summary,  USAF  policy  must  be  considered  in  light  of  the  three 
elements  of  quality,  i.e.,  quality  of  design,  quality  of  conformance  to 
design  and  conformance  to  contractor  management  system  requirements.  The 
main  effort  is  directed  toward  assuring  conformance  of  product  through 
verifying  conformance  of  management  systems.  The  policy  that  relates  to 
quality  of  design  is  limited  and  unclear  in  intent.  Quality  policy  also 
excludes  time  related  product  conformance  to  design,  i.e.,  reliability 
requirements.  Policy  is  stated  in  terms  that  are  undefined  such  as  quality 
characteristics,  contractual  technical  system  quality  requirements,  quality 
requirements,  quality  system  requirements,  quality  and  technical  requirements, 
essential  quality  requirements,  product  characteristics,  quality  deficiencies, 
deficiencies,  etc.  Clarification  and  consistency  in  terms  would  improve 
quality  policy.  Quality  policy  may  also  be  clarified  if  clear  recognition 
and  distinction  is  made  between  product  quality  assessments  and  quality 
management  system  assessments. 


Reliability  policy  depends  less  on  the  contractor's  management 
system  and  emphasizes  system  performance  in  reliability  terms.  Reliability 
policy  establishes  a  requirement  for  equipment  reliability  performance 
records^  Reliability  policy  influences  quality  of  design  by  establishing 
reliability  goals  and  periodically  reviewing  them  at  upper  management  levels. 
There  is,  by  comparison,  no  such  review  of  non-reliability  or  other  quality 
performance  goals.  Reliability  policy  takes  into  account  other  contractual 
incentives  such  as  warranties.  Policy  also  requires  an  organized  effort  to 
collect  and  analyze  reliability  and  maintainabil ity  data  on  AFSC  systems 
at  the  Division  level.  Policy  also  requires  an  interface  between  the  Division 
and  the  IOT&E  and  DT&E  efforts  and  a  related  data  bank  of  findings. 

In  a  comparison  of  policy,  reliability  policy  may  be  seen  to 
generate  a  high  level  of  visibility  based  almost  exclusively  on  performance 
of  systems.  Quality  policy  does  not  concentrate  on  and  highlight  product 
quality,  but  rather  directs  more  management  effort  toward  administrative 
controls,  resulting  in  a  lower  management  level  of  concern  and  visibility. 
Quality  policy  would  be  improved  if  it. were  revised  to  provide  additional 
emphasis  on  product  quality,  as  opposed  to  management  system  quality. 
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VI.  USAF  QUALITY  ASSURANCE  POLICY  AND  MOTIVATIONAL  VOIDS 


VI.  USA F  QUALITY  ASSURANCE  POLICY  AND  MOTIVATIONAL  VOIDS 


A.  GENERAL 

The  USAF  Quality  Assurance  Program,  while  virtually  silent  on 
motivational  conceptual  planning  and  utilization  of  motivational  concepts, 
nevertheless  does  contain  two  subtle  and  strong  motivational  aspects. 

The  first  is  that  approval  to  pay  the  firm  is  normally  predicated  on 
acceptance  of  the  material  after  it  is  inspected  to  assure  that  it  meets 
all  contract  requirements  (DAR  Appendix  I  1.-101  (b).  The  second  is  the 
policy  that  contracts  should  not  be  awarded  to  contractors  with  a  history 
of  providing  products  or  services  of  an  unsatisfactory  quality  (AFR  74-1. 
4.d). 


The  Quality  Assurance  Policy  is  buttressed  in  other  parts  of  the 
DAR  where  contractor  hi  story/ past  performance  is  to  be  a  factor  in  awarding 
contracts  and  establishing  profit  levels  when  profit  is  negotiated.  In 
theory,  these  features  of  USAF  procurement  methodology  should  be  adequate 
motivational  devices  in  themselves.  There  are,  however,  other  considerations 
such  as  the  characteristics  of  the  Defense  market  which  impact  on  the 
effectiveness  of  these  basic  motivators. 

Government  procurement  methodology  does  not  mesh  as  simply  with 
the  market  as  the  free  market  principles  permit.  This  is  a  consequence  of 
several  factors  including  the  public  responsibilities  of  the  buyers,  the 
wide  range  of  products  and  services  purchased,  the  variety  of  procurement 
situations,  methodology  changes  resulting  from  changes  in  top  level  manage¬ 
ment  and  over-reactions  to  errors  that  occur.  The  DAR  itself  is  profit 
centered  with  penalty  features  and  is  based  on  the  instant  contract  without 
serious  attention  to  development  of  a  corporate  approach  which  could  be 
used  in  overall  procurement  strategies  and  particular  contracting/negotiating 
situations. 


Current  USAF  Quality  Policy  is  management  system  oriented,  has  a 
primary  emphasis  on  the  production  phase  of  the  life  cycle,  conveys  no 
sense  of  urgency  to  suppliers  and  is  lacking  in  overt  positive  incentive 
methodology.  Major  contractors  are  not  motivated  to  respond  to  the  threat/ 
reward  stimuli  inherently  associated  with  the  acceptance  act.  Specific 
opportunities  to  improve  the  motivational  aspects  and  effectiveness  of 
quality  assurance  management  are  available  and  in  some  cases  they  require 
additional  emphasis  or  changes  to  some  paragraphs  of  the  DAR.  Paragraphs 
VI  B,  VI  C  and  VI  D  represent  three  major  opportunities  to  improve  Quality 
Assurance  Policy  and  are  set  forth  below. 

B.  CONTRACTOR  QUALITY  DATA  -  VOID  #1 

DoDD  4155.1  and  AFR  74-1  both  establish  the  principle  that 
contracting  officers  will  maintain  historical  quality  data  that  will  be 
used  to  preclude  the  award  of  contracts  "to  prospective  contractors  with 
a  previous  JjJ story,  of  providing  products  or  services  of  an  unsatisfactory 
quality."  In  the  case  of  the  AFSC,  the  application  of  this  requirement 
may  be  less  appropriate  since  the  AFSC  retained  plants  are  all  large 
contractors  with  a  lengthy  record  of  supplying  material  to  the  USAF  and 
other  customers.  DAR  3-401 (b)  advises  contracting  personnel  that  their 
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objective  should  be  to  ensure  that  profits  should  be  directly  relatable 
to  performance,  including  reliability  and  quality. 

A  quality  history  of  these  large  corporations  incorporating  all 
elements  of  the  quality  triad  could  be  useful  in  planning  for  future  or 
newly  established  development  or  full-scale  production  programs.  Such  a 
quality  history  of  a  firm  is  more  likely  to  be  positive  than  negative  and 
will  undoubtedly  illustrate  more  strength  than  weaknesses.  There  are  two 
areas  of  information  that  are  recognized  by  some  personnel,  but  traditionally, 
are  not  organized  and  available  at  crucial  procurement  milestones.  These 
information  areas  are  twofold:  (1)  the  corporation's  general  management  approach 
to  quality,  i.e.,  the  relative  position  of  quality  issues  in  the  hierarchy  of  all 
corporate  goals  and  issues;  and  (2)  the  complete  quality  histories  of  systems  as 
they  progressed  through  the  life  cycle.  If  this  information  is  organized  and 
recorded  systematically  on  a  routine  periodic  cycle,  insight  into  corporate 
reactions  to  quality  issues  can  be  utilized  advantageously  in  early  as  well  as 
late  life  cycle  negotiations.  The  information  would  improve  parametric  analyses 
capabilities  and  permit  better  planning  and  allocation  of  resources. 

There  is  a  void  in  that  there  is  no  USAF  corporate  memory  providing 
an  organized  historical  perspective  of  a  contractor's  quality  performance  in 

developing  and  producing  major  systems. 

Appendix  VI— I ,  Contractor  Quality  Performance  Evaluation  Report, 
will  begin  the  process  of  eliminating  this  void.  It  will  provide  quality 
assurance  organizations  with  an  opportunity  to  capitalize  on  the  motivation 
that  occurs  when  the  firm  realizes  the  three  elements  of  quality  performance 
on  a  current  contract  will  be  examined  when  a  future  proposal  is  considered  r  / 
for  award.  It  is  recommended  that  it  be  adapted  and  that  the  CMD  be  made 
responsible  for  collecting,  evaluating  and  providing  the  reports  to  contracting 
officers.  Appendix  VI-2  is  implementing  language  proposed  for  inclusion  in 
AFR  74-1  and  AFSCR  74-1. 

It  is  recommended  that  the  contractor  quality  performance  report 
procedures  be  adapted  on  a  trial  basis.  The  adaption  of  this  procedure 
will  provide  an  opportunity  to  supply  contractor  quality  history  in  an 
organized  analytical  format  for  the  use  of  those  officials  involved  in 
contracting  negotiations. 

C.  INCENTIVE  CONTRACTS  AND  PRODUCT  QUALITY  -  VOID  #2 

The  Cost  Plus  Award  Fee  (CPAF)  approach  was  a  subject  of  special 
interest  to  many  persons  interviewed  during  this  investigation,  both 
government  officials  and  contractor  personnel.  The  Award  Fee  approach 
which  permits  subjective  evaluations  as  a  basis  for  fee  determination 
is  applicable  to  considering  excellence  of  effectiveness  of  contractor 
management  systems  required  and  defined  by  the  government,  e.g.,  MIL-Q-9858. 

The  Award  Fee  concept  has  the  dual  motivational  capability  of 
providing  psychological  rewards  to  people  at  lower  levels  of  management  by 
drawing  attention  to  their  activities  while  simultaneously  increasing  corporate 
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profits.  It  also  improves  communications  by  requiring  a  specific  judgment 
of  the  program  in  question  and  by  permitting  the  contractor  to  examine  the 
judgment.  A  last  advantage  is  that  it  results  in  more  considerations  and 
sharper  definitions  of  government  objectives.  The  disadvantage  is  that 
it  entails  a  significant  administrative  burden.  The  burden  arises  because 
it  will  be  necessary  to  involve  numerous  personnel  at  more  than  one  level 
of  government  management.  The  administrative  requirements  resulting  from 
a  careful  evaluation  and  fee  determination  process  does  provide  protection 
against  arbitrary  or  capricious  evaluations. 

The  DAR  does  permit  an  alternative  to  the  CPAF  approach  in  that 
it  is  permissible  to  use  Performance  Incentives,  not  only  in  terms  of 
equipment  performance,  but  also  in  terms  of  the  performance  of  the  firm. 
Although  this  approach  appears  to  have  merit,  it  was  not  explored  in  detail, 
due  to  the  current  emphasis  on  the  CPAF  approach.  It  is  recommended  that 
some  thought  be  given  at  a  future  date  to  considering  performance  incentives 
as  another  viable  quality  assurance  management  concept. 

There  is  a  void  in  that  no  methodology  has  been  developed  to 
incorporate  the  profit  motivation  in-  the  USAF  Qua! ity  Assurance  Program.  ' 

In  view  of  the  current  USAF  field  and  contractor  interest  in  the 
CPAF  method,  it  was  decided  to  develop  an  approach  which,  while  certainly 
not  unique  to  methods  used  by  the  DoD  and  other  Departments,  would  provide 
a  basic  procedure  specifically  applicable  to  product  quality  issues.  This 
approach  was  taken  even  though  during  the  term  of  this  contract,  AFSC 
instructions  were  issued  which  directed  the  use  of  the  award  fee  in  the 
quality  area.  In  view  of  the  findings  resulting  from  the  exploration  of 
this  technique  it  appears  that  the  findings  will  affirm  the  AFSC  decision 
and  there  will  be  benefits  to  be  gained  from  providing  a  methodology  for 
consideration. 

Appendix  VI-3  provides  a  brief  discussion  of  CPAF  contracting 
considerations  and  a  format  which  may  be  applied  in  a  contract  as  a 
specific  quality  award  fee  provision. 

D.  DAR  MOTIVATION  AND  PRODUCT  QUALITY  -  VOID  #3 

The  DAR,  as  has  been  stressed  previously,  established  the  premise 
that  increasing  profit  will  improve  the  performance  of  the  firm.  The 
unstated  assumption  underlying  this  premise  is  that  in  some  fashion  the 
profit  motivator  will  effect  those  employees  who  influence  product  quality. 
As  also  discussed  earlier  in  this  report,  profit  is  not  necessarily  the 
prime  motivational  consideration  of  the  firm  at  all  times.  Hence,  though 
the  profit  assumption  is  true  in  a  general  sense,  its  validity  in  every 
contract  situation  is  subject  to  question.  The  INTEC  survey  confirmed 
findings  of  other  researchers  that,  while  monetary  considerations  are 
important  to  those  who  can  influence  quality,  there  are  other  needs  more 
complex  and  often  more  important  to  the  firm  than  pure  monetary  goals. 

It  has  also  been  found  that  the  remuneration  of  managers  correlated  well 
with  sales  but  less  closely  with  profit.  For  example,  in  some  corporate 
situations  there  may  be  more  emphasis  on  objectives  that  will  contribute 
to  improving  market  position  and  assuring  future  strength  of  the  firm  and 
less  emphasis  on  attempting  to  maximize  profit  on  a  particular  contract. 
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Having  postulated  that  profit  is  in  most  cases  the  dominant 
motive,  the  DAR  has  devoted  limited  attention  to  other  forms  of  motivation. 
Employee  morale  is  discussed  briefly  in  Section  15  as  are  bonuses,  awards 
and  incentives.  Despite  these  limited  exceptions  there  appears  to  be  two 
primary  assumptions:  (1)  that  profit  is  the  principal  motivational  factor, 
and  (2)  that  the  "contractor"  is  a  finite  being  who  will  be  responsive  to 
contractual  profit  schemes.  These  assumptions  are  gross  over  simplifications 
of  complex  organizational/people  situations.  An  obvious  question  is,  who 
will  get  the  profit  and  who  can  influence  the  quality  of  material  and 
workmanship. 

There  is  a  void  in  t ha the  DAR  does  not  devote  sufficient  attention 
to  motivational  considerations  for  those  non-supervi sory  personnel  and 

junior  supervisors  who  can  directly  influence  product  quality  through  their 

own  craftsmanship  and  attention  to  detail. 

The  DAR  provides  no  significant  guidance  which,  (1)  either 
encourages  the  development  of  programs  which  will  provide  motivation,  or 
(2)  for  sharing  increased  incentive  profits  with  personnel  who  are  directly 
involved  in  defense  programs.  There  is,  however,  limited  guidance  in  the 
DAR  that  contains  the  rudiments  of  such  guidance  which,  if  modified,  or 
added  to,  will  effect  improvement.  Recommendations  for  additions  to  DAR 
1-343  and  DAR  15-205  and  15-309  are  included  as  Appendix  VI-4. 

The  above  voids  and  associated  recommendations  reflect  specific 
methods  for  implementing  and  institutionalizing  significant  changes  to  the 
Air  Force  Quality  Assurance  Program.  They  are  also  intended  to  serve  as 
devices  to  modify  traditional  concepts  and  approaches  to  quality  assurance 
management.  They  do  not  however,  reflect  all  of  the  observations  that  were 
suggested  or  encountered  during  the  course  of  the  investigation.  While  the 
scope  of  this  study  did. not  permit  the  development  of  resolutions  to  all  of 
these  findings,  they  are  identified  in  appendix  VI-5. 
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APPENDIX  VI-1 


<>Ao4(  o  /2>f 

T ^  i  ; 

CONTRACTOR  QUALITY  PERFORMANCE  EVALUATION  SUMMARY 


A  suggested  format,  included  as  Attachment  1  to  this  Appendix, 
is  designed  to  provide  a  uniform  format  for  assessi ng  "tlie^contra'ctor's 
performance  and  to  provide  data  that  can  be  used  to  fulfill  the  requirements 
of  policy  guidance  that  requires  evaluation  and  utilization  of  contractor 
performance  data.  The  recommended  evaluation  summary  is  divided  into  three 
categories:  Quality  of  Quality  Management  Systems,  Quality  of  Product 
Conformance  and  Quality  of  Design.  The  assessments  are,  to  a  significant 
degree,  judgmental .  The  absolute  data  that  is  included  will  assist  in 
providing  a  rationale  for  the  assessments.  That  data  will  also,  if  collected 
over  a  period  of  time,  and  over  a  range  of  products,  provide  a  basis  of 
"normal"  data  which  can  be  used  to  successfully  evaluate  and  efficiently 
administer  a  particular  contract. 

In  order  to  assure  a  thorough  understanding  of  the  evaluation 
process,  the  evaluator  must  prepare  a  written  description  or  rationale  of 
the  factor,  that  caused  the  assignment  of  the  rating  which  is  then  multiplied 
by  a  value  factor  to  arrive  at  an  effectiveness  rating.  It  is  not  possible 
to  provide  standard  rationale  in  the  format  due  to  the  diversity  in  products 
and  contracts.  Accordingly,  each  evaluator  should  discuss  the  attached 
proposed  procedure  with  supervisory  personnel.  The  rationale  must  take 
into  account  the  type  of  equipment  contracted  for,  e.g.,  systems/subsystems/ 
components,  etc. 

The  summary  should  be  prepared  on  each  contract  over  $500,000  at 
the  end  of  the  contract,  or  annually,  if  an  extended  period  is  involved  as 
in  a  system  contract.  The  summary  should  be  discussed  with  the  contractor 
and  a  copy  retained  in  the  plant  for  future  contracting/precontracting 
activities.  A  copy  will  also  be  provided  to  the  contracting  officer  and 
to  the  USAFCMD. 

The  individual  score  for  each  element/characteristic  should  be 
assigned  a  value  in  accordance  with  Table  1: 

Table  1 

Quality  Valuation  Ratings 


Excel  1 ent 

91  - 

100 

Very  Good . 

81 

90 

Good 

71 

80 

Margi nal 

61  - 

70 

Submargi nal 

0 

61 
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ATTACHMENT  1 


RECOMMENDED  CONTRACTOR  QUALITY  PERFORMANCE  EVALUATION  SUMMARY 

Date  _ _ _ 

Contractor  _ _ 

Contract  Number  _ ' _ _ 

Contract  Date  _ _ 

Value  _ _ 

Number  of  Items  _ 

Eval uator 


The  objective  of  the  evaluation  is  to  provide  a  continuing  assessment  of  the 
firm's  quality  program.  The  data  below  provides  a  standard  rational  for 
selected  parts  of  the  program.  The  assessment  is  limited  to  these  key 
parts  to  minimize  and  standardize  the  evaluation.  The -eval uator  must 
assign  a  rating  from  Table  1  which  reflects  an  assessment  of  these  activities. 


Category  1,  Quality  of  Quality  Management  Systems  Factor  =  .6 


El ement 

Criteria  Rating 

Factor  Effectiveness 

A. 

Quality  Program 

MIL-Q-9858, 

x  .2  =  % 

Management 

Para.  3.2 

B. 

Production  Processing 
and  Fabrication 

MIL-Q-9858, 

Para.  6.2 

x  .2  =  % 

C. 

Measuring  and  Testing 
Equi pment 

MIL-Q-9858, 

Para.  4.2 

x  .2  =  % 

Total 

Quality  Management 

System  Score  % 

Category  effectiveness  is  found  by  multiplying  the  rating  times  the  factor  (.2). 
Rating  _  x  .2  =  % 
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RECOMMENDED  CONTRACTOR  PERFORMANCE  EVALUATION  SUMMARY  (Cont'd) 


Category  2,  Quality  of  Product  Conformance 


Factor  =  .2 


The  objective  of  this  evaluation  is  to  provide  a  continuing  assessment  of 
the  conformance  of  products  to  design  requirements.  The  data  below  will 
assist  the  evaluator  in  making  what  is  primarily  a  subjective  assessment. 
It  will  provide  one  uniform  element  of  the  analysis  and  will  provide  a 
continuing  evaluation  of  product  conformance.  The  evaluator  must  assign 
a  score  from  Table  1  which  reflects  an  overall  assessment  of  product 
conformance  to  requirements . 


A.  Acceptance  Testing  Results 

(1)  _ units  completed  acceptance  test  with  no  defects. 

(2)  _ units  required  retest  due  to  defects. 

B.  Manufacturing  Effectiveness 

(1)  _ major  non-conformances  were  accepted, 

(2)  _  _ minor  non-conformances  were  accepted. 


Category  effectiveness  is  found  by  multiplying  the  rating  times  the  factor  (.2). 
Rating  _  x  .2  =  _  % 
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RECOMMENDED  CONTRACTOR  PERFORMANCE  EVALUATION  SUMMARY  (Cont'd) 


Category  3,  Quality  of  Design 


Factor  =  .2 


The  objective  of  the  evaluation  is  to  provide  a  continuing  assessment  of  the 
contractor's  quality  of  design.  The  data  below  will  assist  the  evaluator  in 
making  what  is  primarily  a  subjective  assessment.  It  will  provide  one  uniform 
element  of  the  analysis  and  will  provide  a  continuing  evaluation  of  quality 
of  design.  The  evaluator  must  assign  a  score  from  Table  1  which  reflects 
an  overall  assessment  of  design  quality. 


Engineering  Changes 


(1) 


Number  of  Class  1  changes  accepted. 


(2)  _ Number  of  Class  2  changes  accepted. 

(3)  _  Number  of  Class  1  changes  that  reduced  system  requi rements . 


(4)  _  Number  of  Class  1  changes  submitted  requiring  further  changes 

to  parameters/characteristics  revised  in  earlier  approved 
changes . 


Category  effectiveness  is  found  by  multiplying  the  rating  times  the  factor  (.2). 
Rating  _  x  .2  =  _ % 


OVERALL  QUALITY  PERFORMANCE  EVALUATION  SCORE 

Overall  contractor  score  is  found  by  summing  Category  1  score,  plus  Category  2 
score,  plus  Category  3  score. 

Overall  contractor  quality  rating  =  Category  1  score  _ % 

Category  2  score  _ % 

Category  3  score  _ % 

Total  % 


79 


APPENDIX  VI-2 


RECOMMENDED  ADDITIONS  TO  USAF  INSTRUCTIONS 


1.  AFSCR  74-1  ' 

Add  to  Paragraph  5f: 

Provide  an  analysis  of  data  in  the  Contractor  Quality  Performance 
Reports  Center  to  the  contracting  officer  for  all  prospective  con¬ 
tractors  prior  to  award  on  each  contract  with  a  value  of  over 
$500,000. 


2.  AFSCR  74-1 

Add  new  Paragraph  8f: 

Prepare  a  Contractor  Quality  Performance  Report  at  the  conclusion 
of  each  contract  which  value  exceeds  $500,000.  In  the  event  the 
duration  of  the  contract  exceeds  one  year,  a  report  will  be  prepared 
annually.  All  reports  will  be  provided  to  the  Contractor  Quality 
Performance  Report  Center  at  the  Contract  Management  Division  and  to 
the  Purchasing  Office. 


3.  AFCMDR  74-1 

Add  the  recommended  Contractor  Quality  Performance  Evaluation  Summary 
format,  as  per  Appendix  VI-1,  Attachment  1,  to  AFCMDR  74-1. 
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APPENDIX  VI-3 


QUALITY  AWARD  FEE  APPROACH 


The  CPAF  contract  is  described  in  the  DAR  3-405.5  and  should 
be  reviewed  prior  to  development  of  an  Award  Fee  scheme  for  a  contract. 

In  the  event  that  a  decision  has  been  made  to  include  an  Award  Fee  feature, 
it  is  essential  that  the  Award  Fee  approach  be  defined  in  the  RFP  stage. 

The  RFP  should  include  an  explanation  of  the  Award  Fee  features  and  the 
fee  determining  procedures  that  will  be  employed.  The  RFP  must  also 
identify  the  criteria  that  will  be  used  to. evaluate  the  responsiveness 
of  proposals  to  the  Award  Fee  requi rements .  The  Award  Fee  Determination 
Plan  (AFDP)*  should  be  included  as  a  separate  entity  with  the  RFP  package. 
Considerations  should  be  given  to  allowing  prospective  contractors  to 
provide  recommendations  for  possible  improvements  to  the  plan  as  a  draft 
RFP. 


When  a  contract  is  issued  containing  an  Award  Fee  clause  for 
quality,  it  is  advantageous  to  provide  limited  references  to  the  Award 
Fee  in  the  contract  itself  and  to  retain  the  AFDP  as  a  separate  element. 
This  will  allow  the  flexibility  of  modifying  the  AFDP  without  modifying 
the  basic  contract  and  will  simplify  the  administrative  processes.  The 
contract,  however,  should  provide  some  coverage  regarding  the  Award  Fee 
aspect  of  the  contract  including  the  following: 

1.  The  Base  Fee  amount,  (can  be  zero  but  should  be  substantial, 

and  may  be  influenced  by  other  Award  Fee  aspects  of  the  contract). 

2.  The  Maximum  Award  Fee  which  can  be  earned. 

3.  The  time  intervals  of  evaluation  and  pay  schedule  and  proportion 
which  can  be  paid  in  each  interval. 

4.  A  provision  that  fee  amounts  will  be  determined  unilaterally 
by  the  Fee  Determination  Official  (FDO). 

5.  A  provision  that  FDO  determinations  concerning  Award  Fee  earned 
is  binding  on  both  parties  and  not  subject  to  appeal  under  the 
dispute  clause. 


6.  A  provision  for  prompt  payment  of  fee  earned  after  each  determination 
without  a  need  for  contract  modification. 


7. 

8. 


A  provision  allowing  equitable  adjustment  of  fee 
change  orders  or  other  contract  modifications. 


in  the  event  of  O'" 


A  provision  that  in  the  event  of  contract  cancellation  the  FDO 
determination  relative  to  the  amount  of  Award  Fee  available  will 
be  final  and  not  subject  to  the  disputes  clause. 


*  Abbreviations  shown  in  Exhibit  G,  Page  loi 
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The  contract  should  also  identify  the  AFDP  by  date  and  title 
and  include  a  provision  that  the  FDO  has  a  unilateral  right  to  change 
AFDP  coverage.  However,  it  should  establish  that  the  contractor  will 
receive  notice  of  changes  by  some  specified  number  of  days  prior  to  the 
evaluation  period  in  which  the  changes  will  be  applied. 


The  amount  of  fee  that  can  be  awarded  has  a  basic  limitation 
in  DAR  3-405. 6(c)(2) .  .However,  the  exact  amount  of  the  fee  must  be 
assessed  in  light  of  any  other  incentive  fees  that  may  be  a  part  of 
the  contract.  Under  ordinary  circumstances,  the  total  fee  allowed  for 
a  contract  for  Research  and  Development  is  limited  to  15%  of  the  estimated 
cost  or  10%  in  the  case  of  a  CPFF  contract. 


The  Fee  Determination  Official  and  board  members  must  be  chosen 
with  consideration  of  their  availability  to  devote  adequate  time  to  the 
plan  as  well  as  their  position  in  the  government  hierarchy.  It  is  desirable 
to  have  representatives  from  organizations  outside  of  the  plant  when  the 
plan  is  employed.  In  particular,  representation  from  the  using  command 
is  highly  desirable.  In  the  attached  format,  in  some  cases,  personnel  are 
"identified  by  title.  This  is  only  for  illustrative  purposes.  The  evaluation 
intervals  are  also  examples.  Attachment  1  is  a  format  which  may  be  used  as 
an  Award  Fee  plan. 
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ATTACHMENT  1 


SAMPLE  AWARD  FEE  FORMAT  AND  APPROACH 
AWARD  FEE  DETERMINATION  PLAN  FOR 
Contract  No.  _ with  _ (firm) _ 


Contents 


page 

1  General  Terms  and  Conditions  . 

2  Management  and  Administration  of  the  Award  Fee  Program  ... 

3  Evaluation  Factors  . 

4  Award  Fee  Determination  Methodology  . 

5  Changes  in  Plan  Coverage  . 

APPROVED  BY: 


(signature) _  (date) 

(typed  name) 

(title) 


S3 


1.  GENERAL  TERMS  AND  CONDITIONS 

A.  This  plan  describes  the  method  of  administration  of  the  Award  Fee 

provisions  of  contract  no. _ dated  , 

with  _. _  . 

B.  The  term  of  the  contract  is  from  __ _ 

through  _ . 


1.  The  estimated  cost  of  performing  the  contract  is  $ 

2.  The  base  fee  is  $  _ _ 

3.  The  maximum  Award  Fee  is  $ _ . 

4.  The  estimated  cost,  base  fee  and  Award  Fee  are  subject  to  equitable 
adjustments  on  account  of  changes  or  other  contract  modifications. 

5.  The  Award  Fee  earned  and  payable  will  be  determined  periodically  by 
the  Fee  Determination  Official  in  accordance  with  this  plan. 

6.  Award  Fee  findings  and  decisions  are  not  subject  to  the  Disputes 
clause  of  the  contract. 

7.  The  FDO*  may  unilaterally  change  the  matters  im  this  plan,  as  covered 
in  part  5  and  not  otherwise  requiring  mutual  agreement  under  the 
contract,  providing  the  contractor  receives  notice  of  the  changes 

at  least  _ work  (or  calendar)  days  prior  to  the 

beginning  of  the  evaluation  period  to  which  the  changes  apply. 

C.  The  objective  of  the  Award  Fee  provisions  of  the  contract  is  to  afford 
the  contractor  an  opportunity  to  earn  increased  fee  commensurate  with 
the  achievements  of  optimum  performance  in  pursuit  of  contract 
objectives  and  goals. 

2.  MANAGEMENT  AND  ADMINISTRATION  OF  THE  AWARD  FEE  PROGRAM 

The  following  management  procedure  is  established  for  administering  the 

Award  Fee  provisions  of  the  contract. 

A.  Fee  Determination  Official  (FDO) 

(1  )  The  FDO  is  _ Deputy  Director  SPO _ 

(2)  Primary  FDO  responsibilities  are: 

(a)  determining  the  Award  Fee  earned  and  payable  for  each 
evaluation  period  as  addressed  in  Part  4. 

(b)  changing  the  matters  covered  in  this  plan  as  addressed 
in  Part  5,  as  appropriate. 


*  All  abbreviations  are  contained  in  Exhibit  G. 


84 


B.  Performance  Evaluation  Board  (PEB) 

(1)  The  Chairman  of  the  PEB  is  Director  of  Engineering  SPO _ 

The  following  are  voting  members: 

Using  Command  Representative 
Director  of  Contracting  SPO 

Director  of  Quality  Assurance  SPO  _ _ _ 

Chief  of  Quality  Assurance  AFPRO  _ _ 

Other  _ _ _ 

(2)  The  Chairman  may  recommend  the  appointment  of  non-voting  members 
to  assist  the  Board  in  performing  its  functions. 

(3)  Primary  responsibilities  of  the  Board  are: 

(a)  Evaluate  the  contractor's  performance  in  accordance  with 
the  performance  criteria  set  forth  in  the  contract  . 

(and/or  in  this  plan).  Evaluation  methods  will  include 
but  not  be  limited  to: 

performance  monitor  reports 
•  other  reports  which  provide  useful  data 

(b)  Prepare  a  Performance  Evaluation  Board  Report  for  the  FDO, 
including  the  Board's  findings  and  recommendations  for  each 
evaluation  period  as  addressed  in  Part  4. 

(c)  Periodically  review  the  evaluation  plan  and  its  implementation, 
and  recommend  necessary  changes  in  accordance  with  Part  5. 

C.  Performance  Monitors  (PMs) 

(1)  The  PEB  Chairman  will  assign  a  PM  to  each  performance  area  to  be 
eval uated. 

(2)  Each  PM  will  be  responsible  for  complying  with  the  General 
Instructions  for  Performance  Monitors  in  Exhibit  B,  and  any 
specific  instructions  of  the  PEB  Chairman  including  the  following 
primary  responsibilities: 

Monitoring,  evaluating  and  assessing  contractor  performance  in 
assigned  activities. 

.  Periodically  preparing  a  Performance  Monitor  Report  (PMR)  for 
the  PEB. 

.  Recommending  appropriate  changes  in  this  plan  for  consideration 
in  accordance  with  Part  5. 
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3.  EVALUATION  FACTORS 


Evaluation  requirements  are  attached  as  follows: 


'  Requirements 

Attachment 

A. 

Evaluation  Periods  and 

Maximum  Available  Award 

Exhibit  A 

Fee  For  Each 

B. 

Performance  Factors  and 

Evaluation  Criteria 

Exhibit  B 

C. 

Scoring  Ta^e 

Exhibit  C 

D. 

Award  Fee  Conversion  Chart 

Exhibit  D 

The  percentages  described  in  Exhibit  B,  the  scoring  table  in  Exhibit  C 
and  the  conversion  chart  in  Exhibit  D  are  provided  as  a  typical  methodology  for  a 
general  assessment  of  the  proportion  of  the  available  Award  Fee  that  has  been 
earned.  They  provide  only  sample  guidance  and  a  judgmental  evaluation 
of  contractor  performance  and  should  not  be  considered  to  be  the  results 
of  a  rigorous  mathematical  determination. 

4.  AWARD  FEE  DETERMINATION  METHODOLOGY 

A  determination  of  the  Award  Fee  earned  for  each  evaluation  period  will 
be  made  promptly  by  the  FDO  after  the  end  of  the  period.  The  method. to  be 
followed  in  monitoring,  evaluating  and  assessing  contractor  performance 
during  the  period,  as  well  as  for  determining  the  Award  Fee  earned,  is 
described  below.  Exhibit  E-l  summarizes  the  principal  activities  and 
recommended  schedules. 

A.  The  PEB  Chairman  will  assign  a  PM  for  each  performance  area  to  be 
evaluated  under  the  contract.  PMs  will  be  selected  on  the  basis  of 
their  expertise  relative  to  prescribed  areas  of  performance.  The  PEB 
Chairman  may  change  PM  assignments  at  any  time  without  advance  notice 

to  the  contractor  but  should  notify  the  contractor  as  promptly  as  possible 

B.  The  PEB  Chairman  will  provide  to  each  PM  the  following  documents: 

(1)  A  copy  of  the  pertinent  elements  of  the  contract  and  subsequent 
modifications. 

(2)  A  copy  of  this  plan  along  with  any  changes  made. 

i3)  Sufficient  instructions  to  perform  the  PM  responsibilities. 

(4)  Any  unique  instructions  concerning  PM-assigned  performance  areas. 

C.  PMs  will  monitor,  evaluate  and  assess  contractor  performance  and 
discuss  the  results  with  appropriate  contractor  personnel,  in 
accordance  with  the  General  Instructions  for  Performance  Monitors 
(Exhibit  E-2 ) ,  and  any  other  specific  instructions  or  guidance 
furnished  by  the  PEB  Chairman. 
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D.  PMs  will  submit  monthly  PMRs  and,  if  required,  make  verbal 
presentations  to  the  PEB. 

E.  The  PEB  Chairman  will  consider  PMRs  and  request  and  obtain 
performance  information  from  other  units  or  personnel  that 
may  assist  in  determining  contractor  performance.  He  will 
discuss  reports  and  information  with  PMRs  and  other  personnel 
as  he  deems  necessary. 

F.  Promptly  after  the  end  of  each  month,  except  the  final  month  of  the 
eva1 uation  period,  the  PEB  Chairman  will  meet  with  the  contractor 
and  discuss  overall  performance  during  the  period.  If  requested 

by  the  PEB  Chairman,  PMs  and  other  personnel  involved  in  performance 
evaluations  will  attend  the  meeting  and  participate  in  discussions. 

G.  Promptly  after  the  end  of  each  evaluation  period,  the  PEB  will  meet 
and  consider  all  performance  information  obtained,  summarize 
preliminary  findings,  and  determine  recommendations  for  coverage 

in  the  PEBR. 

H.  Subsequently,  the  PEB  Chairman  will  meet  with  the" contractor  Award 

Fee  representative  and  discuss  preliminary  findings  and  recommendations. 

As  requested  by  the  PEB  Chairman,  PMs  and  other  personnel  involved  in 
performance  evaluation  will  attend  the  meetings  and  participate  in 
discussions.  At  this  meeting,  the  contractor  representative  will  be 
given  an  opportunity  to  submit  matters  in  the  firm's  behalf,  including 
an  assessment  of  performance  during  the  evaluation  period. 

I.  After  meeting  with  contractor  personnel,  the  PEB  will  consider  matters 
presented  by  contractor  personnel  and  establish  Board  findings  and 
recommendations  for  the  PEBR. 

J.  The  PEB  Chairman  will  prepare  the  report  for  the  period  and  submit  it 
to  the  FDO  for  use  in  determining  the  Award  Fee  earned.  The  report 
will  include  a  recommended  range  of  Award  Fee  with  supporting 
documentation.  When  submitting  the  report,  the  Chairman  will  inform 
the  FDO  whether  or  not  the  contractor's  representative  desires  to 
present  any  matters  to  the  FDO  before  the  Award  Fee  determination  is  made. 

K.  The  FDO  will  review  the  PEBR  and  discuss  it  with  the  PEB  Chairman  and 
other  personnel,  as  appropriate.  If  requested  by  the  contractor,  or 

if  the  FDO  considers  it  appropriate,  the  FDO  will  meet  with  the  contractor 
for  discussions.  If  requested  by  the  FDO,  the  PEB  Chairman  and  any  other 
personnel  involved  in  performance  evaluation  may  be  required  to  attend 
the  meeting  with  the  contractor. 

L.  The  FDO  will  determine  the  amount  of  Award  Fee  earned  during  the  period. 
The  amount  determined  will  include  the  results  of  judgemental  assessments 
as  well  as  utilizing  the  analytical  procedures  herein.  The  FDO's 
determination  of  the  amount  of  Award  Fee  earned  and  the  basis  for  this 
determination  will  be  stated  in  the  Award  Fee  Determination  Report  (AFDR). 
The  report  will  be  signed  by  the  FDO  and  provided  to  the  contractor  as 
justification  for  payment  of  the  Award  Fee  granted. 
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5.  CHANGES 


A.  Unilateral  Changes 

The  FDO  may  unilaterally  change  any  matters  covered  in  this  plan 
not  otherwise  requiring  mutual  agreement  under  the  contract,  prior 
to  the  beginning  of  any  evaluation  period  by  timely  notice  to  the 
contractor  in. writing.  The  changes  will  be  made  without  formal 
modification  of  the  contract. 

B.  Method  For  Making  Changes 

The  method  to  be  followed  for  changing  plan  coverage  follows. 

Exhibit  F  summarizes  the  principal  actions  and  schedules 
i nvol ved. 

1.  Any  person  involved  in  the  administration  of  the  Award  Fee 
provisions  of  the  contract  may  recommend  any  changes  in  the 
plan  which  will  result  in  improved  performance  or  improve  the 
Award  Fee  determination  process  by  providing  the  recommended 
changes  to  the  PEB  for  his  evaluation. 

2.  Any  proposed  changes  will  be  coordinated  with  the  contractor 
by  the  PEB  Chairman. 

3.  Prior  to  the  end  of  each  evaluation  period,  the  PEB  will  submit 
changes  applicable  to  the  next  evaluation  period  for  approval 

by  the  FDO  with  appropriate  comments  and  justification,  or  inform 
the  FDO  that  no  changes  are  recommended  for  the  next  period. 

4-  * _ work  days  before  the  beginning  of  each  evaluation 

period,  the  FDO  will  notify  the  contractor  in  writing  if  any 
changes  w i 1 1  be  made.  If  any  such  notification  is  not  provided 
to  the  contractor  within  the  agreed-to  number  of  work  days  before 
the  beginning  of  the  next  period,  existing  plan  coverage  will 
continue  in  effect  for  the  next  evaluation  period. 


*  Numbers  of  days  are  to  be  agreed  upon  by  both  parties  during  contract 
negotiations . 
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EXHIBIT  A  to  AFDP  FOR  CONTRACT  # 
Contract  No.  with 


EVALUATION  PERIODS  AND  MAXIMUM 
AVAILABLE  AWARD  FEE  FOR  EACH 


Eval uation  Period 

Max.  Available 

No .  Duration*  Endi nq  Award  Fee 

1  3  months 

2  6  months 

3  6  months 

4  6  months 

5  6  month  intervals  to 

end  of  contract 


Maximum  Fee  Each  Evaluation  Period  =  Total  Fee 

Number  of  evaluation  periods 


Periods  and  equal  fee  distributions  here  are  examples  only.  If  the 
fee  is  varied  for  different  periods,  the  evaluation  equation  must  be 
modi fied . 
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EXHIBIT  B  TO  AFDP  FOR  CONTRACT  #  _ _ _ 

PERFORMANCE  FACTORS  AND  EVALUATION  CRITERIA 


The  performance  areas  to  be  evaluated  are 

identifed  below 

The  evaluation  criteria  for  each  area  are 
i ndicated. 

attached,  as 

Area  No. 

Factor  Factor 

Identification  Weight 

See 

Exhibit 

1 

Quality  System  .25 

Management 

B-l 

2 

) 

Production  of  Non-  .25 

conforming  material 

B-2 

3 

Major  Assemby  Ins  pec-  .25 

tion  Findings 

-  B-3 

4 

Completed  End  Item  .25 

Quality  Assessment 

B-4 
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EXHIBIT  B-l  TO  AFDP  FOR  CONTRACT  #  _ 

EVALUATION  CRITERIA  FOR  PERFORMANCE  FACTOR  NO.  1 

QUALITY  SYSTEM  MANAGEMENT  Factor  Weight  .25 


Description  of  Element: 

Para.  3  MIL-Q-9858A 

The  objective  of  the  element  is  to  encourage  corporate  wide  involvement 
in  the  quality  management  system. 

Sub-Elements  to  Consider: 

Para.  3.1  Organization,  3.4  Records,  3.5  Corrective  Action,  3.6  Costs 
related  to  quality. 

Eva! uation  Criteria:  ^ 


Evaluation  will  be  per  MIL  Handbook  H50,  Para.  3.1,  3.4,  3.5  and  3.6  except 
that  complete  cost  data  should  be  available  for  prevention,  appraisal,  and 
failure,  (both  internal  and  external  costs). 

Basis  or  Standard  for  Measuring  Performance: 

This  is  a  subjective  appraisal  of  a  firm's  performance  as  evaluated  by  the 
PM.  It  is  to  be  used  in  consonance  with  the  narrative  instructions  in  the 
rating  table,  (Exhibit  C). 

Sub- Element  Weights: 


3.1  - 

.3 

Para. 

3.1 

Score 

x  .3  = 

3.4  - 

.2 

Para.  3.4 

Score 

x  .2  = 

3.5  - 

.3 

Para. 

3.5 

Score 

x  .3  = 

3.6  - 

.2 

Para.  3.6 

Score 

x  .2  - 

Total 


91 


EXHIBIT  B-2  TO  AFDP  FOR  CONTRACT  # 


EVALUATION  CRITERIA  FOR  PERFORMANCE  FACTOR  NO.  2 


NON-CONFORMING  MATERIAL  Factor  Weight  .25 
Description  of  Element: 

The  objective  of  this  factor  is  to  develop  a  manufacturing  process  in 
which  the  production  of  defective  material  is  the  consequence  of  onlv 
random  events. 


Sub-Element  to  Consider: 

(1)  Purchased  material,  (2)  machine/process  functions,  (3)  assembly, 
(4)  Productivity 


Evaluation  Criteria: 

1.  Is  there  an  approved  overall  company  cost  factor  for  manufacturing 
losses  resulting  from  costs  associated  with  the  procurement  of 
defective  material? 


2.  Are  man/machine  process  capability  studies  performed  to  analyze 
production  problems? 

3.  Are  process  control  records  maintained  at  assembly  stations  that 
identify  recurring  defects? 

4.  Is  there  a  positive  program  to  publicize  improvements  in 
productivity  resulting  from  quality  improvements? 

Basis  or  Standard  for  Measuring  Performance: 


This  is  a  subjective  appraisal  of  the  firm's  performance  as  evaluated  by 
the  PM.  It  is  to  be  used  in  consonance  with  the  narrative  instructions 
in  the  rating*  table  (Exhibit  C). 

Sub-Element  Weights: 


1 . 

-  .25 

Score 

X 

.25  = 

2. 

-  .25 

Score 

X 

.25  = 

3. 

-  .25 

Score 

X 

.25  = 

4. 

-  .25 

Score 

X 

.25  = 

Total 
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EXHIBIT  B-3  TO  AFDP  FOR  CONTRACT  #  _ 

EVALUATION  CRITERIA  FOR  PERFORMANCE  FACTOR  NO.  3 

MAJOR  ASSEMBLY  INSPECTION  Factor  Weight  .25 

Description  of  Element:. 

The  objective  of  this  element  is  to  assure  the  development  of  a  system 
to  analyze  and  eliminate  non-random  manufacturing/assembly/test  problems 
that  contribute  to  end  item  testing  problems,  delays  and  costs. 

Sub-Elements  to  Consider: 

All  areas  where  subsystems  or  major  components  are  given  a  final  checkout 
prior  to  assembly  into  the  final  item/system  configuration.  In  the 
interests  of  focusing  attention  sub-elements  should  not  exceed  10. 

Evaluation  Criteria: 

The  criteria  should  concentrate  on: 

1.  identification  of  significant  problems 

2.  analysis  as  to  their  cause 

3.  promptness  and  effectiveness  of  corrective  action  taken 
Basis  for  Measuring  Performance: 

This  is  a  subjective  appraisal  of  the  firm's  performance  as  evaluated  by 
the  PM.  It  is  to  be  used  in  consonance  with  the  narrative  instructions 
in  the  rating  table  (Exhibit  C). 

Sub-Element  Weights: 

The  evaluation  must  determine  how  many  sub-elements,  (subsystem  checkout 
stations)  there  are.  It  is  recommended  that  an  equal  weight  be  assigned 
to  each  and  that  a  table  and  scoring  procedure  similar  to  that  in 


Exhibit  B-l 

be  prepared,  e.g.  -  * 

sub-el ement 

1  -  .33 

Score 

x  .33  = 

sub-el ement 

2  -  .33 

Score 

x  .33  = 

sub-el ement 

3  -  .33 

Score 

x  .33  = 

Total 


*  The  number  of  elements  is  immaterial  but  the  total  score  cannot  exceed  1.0. 
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EXHIBIT  B-4  TO  AFDP  FOR  CONTRACT  #  _ 

EVALUATION  CRITERIA  FOR  PERFORMANCE  FACTOR  NO.  4 

COMPLETED  END  ITEM  QUALITY  ASSESSMENT  Factor  Weight  .25 


Description  of  Element: 

The  objective  of  this  element  is  to  generate  a  measure  of  product 
quality  in  terms  of  conformance  with  design  requirements .  It  provides 
product  conformance  information  both  in  the  manufacturer' s  facility 
and  after  delivery. 

Sub-Elements  to  Consider: 

1.  Number  of  defects  found  by  contractor  in  final  end  item  inspection/test 
on  each  end  item. 

2.  Number  of  minor  waivers  granted  on  each  end  item.  - 

3.  Number  of  major  waivers  granted  on  each  end  item. 

4.  Number  of  defects  found  by  user  if  inspection  is  performed. 

5.  Number  of  user  generated  deficiency  reports. 

Evaluation  Criteria: 

In-plant  final  inspection 
User  inspection  activities 
User  routine  deficiency  reports 

Basis  or  Standard  for  Measuring  Performance: 

This  is  a  subjective  appraisal  of  the  firm's  performance  as  evaluated 
by  the  PM.  It  is  to  be  used  in  consonance  with  the  narrative  instructions 
in  the  rating  table.  (Exhibit  C). 

Sub-Element  Weights: 


1  - 

.2 

Sub-el ement 

1 

Score 

x  .2  = 

2  - 

.1 

Sub-el ement 

2 

Score 

x  .1  = 

3  - 

.3 

Sub-el ement 

3 

Score 

x  .3  = 

4  - 

.3 

Sub-el ement 

4 

Score 

x  .3  = 

5  - 

.1 

Sub-el ement 

5 

Score 

X  .1  = 

Total 


Narrative  Grade 

Excel  1 ent 

Good 

Satisfactory 

Margi nal 

Unsati sfactory 


EXHIBIT  C  TO  AFDP  FOR  CONTRACT  # 


RATING  TABLE 


Proportion  of  Maximum 

Description  Achievable  Score  Available 


Performance  far  exceeds  that 
normally  achieved  for  a'  similar 
task.  Few  errors  are  made. 
Management  and  employee  controls 
outstanding. 

Performance  exceeds  that 
normally  achieved.  Above 
average  achievement  of  all 
areas  of  element. 

Performance  meet  minimum 
standards.  Mo  exceptional 
management  emphasis  on  quality, 
problems  encountered,  routinely 
resol ved . 

Performance  less  than  expected. 
Continuing  extraordinary  efforts 
required  to  assure  material  is 
meeting  requirements. 

Performance  not  acceptable. 
Material  cannot  be  shipped  with¬ 
out  government  inspection. 
Quality  management  system 
not  acceptable. 


90-100 


60  -  90 


30  -  50 


10  -  20 


0 
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EXHIBIT  D  TO  AFDP  FOR  CONTRACT  § 


The  following  table  converts  achieved  Award  Fee  scores  into  fee  earned. 
Performance  Scores : 


Factor 

No.  1 1 

Score 

X 

.25 

=  Available 

% 

of  Award 

Fee 

Factor 

No.  2, 

Score 

X 

.25 

=  Available 

% 

of  Award 

Fee 

Factor 

No.  3, 

Score 

X 

.25 

=  Available 

% 

of  Award 

Fee 

Factor 

No.  4, 

Score 

X 

.25 

=  Available 

% 

of  Award 

Fee 

Total  %  of  available  Award  Fee  earned 


Total  %  of  Award  Fee  earned  _ x  available  fee  _ =  _ fee  earned. 
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EXHIBIT  E-l  TO  AFDP  FOR  CONTRACT  # 


The  following  is  a  summary  of  the  principal  actions  involved  in 
determining  the  Award  Fee  for  each  evaluation  period. 


Schedule 

Action  (Workdays ) 

PEB  Chairman  appoints  PMs,  provides 
necessary  guidance,  and  informs 

contractor  as  required 

PMs  assess  performance  and  discuss  results 

with  contractor  throughout  the  period 


PMs  submit  PMRs  to  PEB 

PEB  obtains  performance  information  from 
other  procuring  activity  personnel 

PEB  considers  PMRs  and  any  other  available 
performance  information 

PEB  Chairman  discusses  overall  performance 
with  contractor  during  period 


PEB  meets  and  prepares  preliminary  findings 
and  position  for  PEBR 


last  day  of  each  month 


as  necessary 


continui ng 


_  days  after  end  of  each 

month,  except  last  month  in 
period 


_  days  after  end  of 

period 


PEB  Chairman  meets  with  contractor  and  discusses 

preliminary  findings  and  position  _ days  after  end  of 

,  period 

PEB  establishes  findings  and  recommendations 

for  PEBR  _  day s  after  end  of 

period 


PEB  Chairman  submits  PEBR  to  FDO 


FDO  reviews  PEBR  and  discusses  it  with  PEB, 
as  appropriate 


FDO  sends  AFDR  to  contractor  and  contracting 
office 


_  days  after  end  of 

period 


_  days  after  end  of 

period 


_  days  after  end  of 

period 


The  PEBC  will  establish  appropriate  lists  of  subsidiary  actions  and 
schedules  to  meet  the  above  schedules,  as  necessary. 
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EXHIBIT  E-2  TO  AFDP  FOR  CONTRACT  It  _ 

PERFORMANCE  MONITORS  INSTRUCTIONS 


I.  Monitoring  and  Assessing  Performance 

a.  PMs  will  prepare  outlines  of  their  assessment  plans  as  a  part  of  the 
PEB  documentation.  The  plan  should  be  discussed  with  appropriate 
contractor  personnel  to  assure  understanding  of  the  evaluation  and 
assessment  methodology. 

b.  PMs  will  plan  and  carry  out  both  scheduled  and  unannounced  assessment 
visits;  however,  before  each  scheduled  visit,  the  PM  should  contact 
appropriate  contractor  personnel  who  may  accompany  the  PM,  if  desired. 

c.  PMs  will  conduct  all  assessments  in  an  open,  objective  and  cooperative 
spirit  so  that  a  fair  and  valid  evaluation  is  achieved.  This  will 
enable  contractor  personnel  to  plan  improvements  in  performance. 
Positive  performance  accomplishments  must  be  recognized  as  well  as 
inadequate  results. 

d.  After  assessments,  the  PM  will  discuss  the  results  with  appropriate 
contractor  personnel,  noting  any  observed  deficiencies  and/or 
accompanying  recommendations.  Areas  of  poor  performance  will  be 
discussed  to  afford  the  contractor  an  opportunity  to  clarify  possible 
misunderstandings  and  to  resolve  deficiencies. 

e.  PM  contacts  with  contractor  personnel  are  to  be  accomplished  within 
the  context  of  official  contractual  relationships.  Any  activity  or 
association  which  might  cause,  or  give  the  appearance  of  causing, 

a  conflict  of  interest  must  be  avoided. 

f.  PM  discussions  with  contractor  personnel  are  not  to  be  used  to  instruct 
or  direct  contractor  personnel  in  the  performance  of  a  contract. 


2.  Documenting  Evaluations/Assessments 

Evaluations  and  assessments  conducted,  results  obtained,  and  discussions 
with  contractor  personnel  will  be  documented  immediately  after  each  such 
actions  by  preparing  a  brief  summary  of  observations  and  discussions  with 
contractor  personnel . 

3.  Evaluation/Assessment  Reports 

PMs  will  prepare  a  formal  PMR  in  accordance  with  the  following  instructions 
and  submit  it  to  the  PEB,  or  others,  if  appropriate,  at  the  end  of  each 
month.  Information  essential  to  the  PMR  element  involved  should  be 
identified,  and  a  format  specified  if  deemed  advisable. 
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4.  Verbal  Reports 

PMs  will  be  prepared  to  make  verbal  reports  as  required  by  the  PEB 
Chairman. 

5.  Performance  Monitor  Report 

The  PMR  will  contain,  as  a  minimum,  a  record  of  each  meeting  or  discussion 
in  connection  with  the  Award  Fee  reviews,  including  the  names  of  contractor 
personnel,  date  of  meeting(s),  and  a  brief  summary  of  discussions.  Any 
differences  of  opinion  with  contractor  personnel  must  be  included  with  a 
fuTl  and  complete  discussion  of  the  issues. 
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EXHIBIT  F  TO  AFDP  FOR  CONTRACT  #  _ 

CONTINGENCY  PLAN  FOR  CHANGING  PLAN  COVERAGE 


The  following  is  a  summary  of  the  principal  actions  involved  in 
changing  plan  coverage. 


Schedul e 

Action  (Workdays ) 


1 . 

PEB  drafts  proposed  changes 

as 

necessary 

2. 

PEB  coordinates  proposed  changes 
with  contractor 

as 

necessary 

3. 

PEB  submits  recommended  changes  to  FDO 

days  prior  to  end 

of 

each  period 

4. 

FDO  notifies  contractor  of  changes  or 
that  there  are  no  changes 

days  before  start  of 

applicable  period 
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EXHIBIT  G  -  LIST  OF  TERMS  AND  ABBREVIATIONS 


The  following  is  an  alphabetical  list  of  all  acronyms  used  throughout 
Appendix  VI-3: 


AF 

Award  Fee 

AFDP 

Award  Fee  Determination  Plan 

AFDR 

Award  Fee  Determination  Report 

CAFR 

Contractor  Award  Fee  Representative 

CPAF 

Cost-Plus  Award  Fee 

FDO 

Fee  Determination  Official 

FM 

Function  Monitor 

FMR 

Function  Monitor  Report 

FPI 

Fixed-Price  Incentive 

ICC 

Incentive  Contracting  Committee 

PEB 

Performance  Evaluation  Board 

PEBC 

Performance  Evaluation  Board  Chairman 

PEBR 

Performance  Evaluation  Board  Report 

PEC  ■ 

Performance  Evaluation  Committee 

PECR 

Performance  Evaluation  Committee  Report 

PM 

Performance  Monitor 

PMR 

Performance  Monitor  Report 

RFP 

Request  for  Proposal 
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DAR  MOTIVATION  AND  QUALITY 


The  following  proposed  changes/additions  to  the  DAR  are 
recommended  as  procurement  policy  designed  to  improve  methods  for 
motivating  contractor  employees  to  improve  the  quality  of  material 
produced  for  the  USAF.-  The  changes  achieve  this  by:  (1)  encouraging 
cash  bonuses  and  other  awards  for  non-management  personnel,  and  (2) 
by  encouraging  the  development  of  effective  organized  motivational 
programs .  . 


Add:  1-343  -  Contractor  Employee  Motivation 


It  is  recognized  that  profit  is  the  basic  motive  of  the  business 
enterprise  and  accordingly,  the  Department  of  Defense  utilizes  the  profit 
motive  to  stimulate  efficient  performance  by  contractors  (DAR  3-808). 

The  stimulation  of  businesses  to  encourage  efficiencies  through 
employee  recognition  and  reward  for  superior  performance  has  traditionally 
been  considered  as  an  internal  business  management  prerogative  and  has  not 
been  addressed.  Typically,  when  firms  provide  bonuses  or  other  rewards  for 
outstanding  performance,  it  is  restricted  to  management  personnel.  There 
has  also  been  limited  guidance  provided  which  encouraged  the  development 
of  organized  and  officially  recognized  motivational  programs . 


In  order  that  all  avenues  be  explored  for  maximum  production  and 
performance  efficiency,  from  both  the  quality  and  quantity  viewpoint,  the 
Department  of  Defense  recommends  that  contractors  give  serious  consideration 
to  the  establishment  of  incentive  policies  to  adequately  compensate  superior 
and  outstanding  employee  performance  through  the  payment  of  bonuses  and/or 
other  material  rewards.  These  policies  which  now  include  management 
personnel  should  be  extended  to  non-supervi sory  and  junior  management 
personnel.  It  is  further  recommended  that  considerations  be  given  to  the 
development  of  carefully  designed  and  effective  motivational  programs  intended 
to  improve  the  quality  of  material  through  improving  the  efficiency  and  morale 
of  the  work  force. 


Add :  1  .343 . 1  -  Pol i cy 


It  is  the  policy  of  the  Department  of  Defense  to  recognize  that 
is  to  the  ultimate  benefit  of  the  government  for  contractors  to  stimulat 
internal  efficiencies  through  the  utilization  of  organized  motivational 
programs  as  well  as  payment  of  bonuses  and/or  other  material  rewards  to 
its  employees  for  superior  performance.  Such  payments  will  be  consider 
as  allowable  items  of  cost  under  the  provisions  of  DAR  Section  15. 


Add:  15-205,6(1)  -  Bonuses  and  Other  Rewards  for  Outstanding  Performance 

Bonuses  and  other  material  rewards  paid  by  the  contractor  to  its 
employees  as  additional  compensation  (over  and  above  regular  wages)  for 
superior  or  outstanding  performance  are  allowable^  if  reasonable  and  in 
accordance  with  established  policies  of  the  contractor. 
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Revise:  15-309.11  -  Employee  Morale,  Health  and  Welfare  Costs  and  Credits 


The  costs  of  house  publications,  health  or  first-aid  clinics 
and/or  infirmaries,  recreational  activities,  employees'  counseling  services, 
and  other  expenses  incurred  in  accordance  with  the  institution's  established 
practice  or  custom  for  the  improvement  of  working  conditions,  employer/ 
employee  relations,  employee  morale,  and  employee  performance,  are  allowable.. 
The  costs  of  motivational  programs  designed  to  improve  the  efficiency  of 
employee  performance  and  to  improve  the  quality  of  material  “are  allowable 

and  are  encouraged./ .  Income  generated  from  any  of  these  activities 

will  be  credited  to  the  cost  thereof  unless  such  income  has  been  irrevocably 
set  over  to  employee  welfare  organizations.' 
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No  public  awareness  of  quality  successes 

No  centralized  approach  to  motivation 

No  sense  of  urgency 

DAR  assumes  profit  relates  to  quality 

Contractor  performance  not  a  Weighted  Guideline  Factor 

No  quality  consideration  in  procurement  plan  (DAR  1-2100) 

System  quality  other  than  reliability  not  emphasized 

DAR  14  does  not  interface  with  other  DAR  parts 

Primary  quality  emphasis  is  on  management  system 

Quality  of  design  and  quality  of  product  policy  needs  reassessment 

No  significant  conceptual  change  to  quality  assurance  in  years 

Quality  modifiers  confuse  definitions 

Productivity  aspect  of  quality  needs  highlighting 

Poor  specifications  are  seen  by  contractors  as  serious  quality  problem 
No  DoD  reliability  focal  point 

Penalty  clauses  such  as  C.O.D.  have  strong  negative  motivational  implications 

No  centralized  orchestration  of  positive  and  negative  motivational  elements 
of  existing  policy 

Excessive  management  controls  may  relegate  contractor  responsibility  to 
theory  only 


Quality  Management  Observations 
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VII.  FINDINGS,  CONCLUSIONS  AND  RECOMMENDATIONS 


VII.  FINDINGS,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  BACKGROUND 

This  report  presents  the  findings  of  an  investigation  performed 
to  examine  United  States  Air  Force  contracting  and  acquisition  techniques 
utilized  to  assure  the  quality  and  reliability  of  weapon  systems.  It  was 
performed  by  the  International  Technology  Corporation  (INTEC)  at  the  request 
of  the  United  States  Air  Force  Business  Research  Management  Center.  The 
principal  objective  of  the  study  was  to  examine  the  general  motivational 
aspects  of  the  USAF  contracting  techniques  with  particular  attention  to 
quality  assurance  and  reliability  policies,  management  techniques  and 
practi ces . 


The  study  methodology  consisted  of  three  phases:  an  examination 
of  current  qual i ty/rel iabil i ty  policy;  interviews  with  USAF  procurement  and 
contract  administration  officials  and  industry  officials;  and  a  wide-guaged 
industry  questionnaire.  The  questionnaire  was  designed  to  obtain  an 
industry-wide  viewpoint  of  quality/rel iabil ity  policy.  Existing  DoD  policy 
was  reviewed  and  a  comprehensive  literature  search  was  undertaken  to  identify 
other  sources  of  information  in  the  product  quality  motivational  arena. 

Fol lowing. completion  of  the  interviews,  literature  search,  examination  of 
the  questionnaire  results  and  policy  review,  the  results  were  integrated 
to  determine  what,  if  any,  recommendations  should  be  made. 

B.  FINDINGS 

There  are  two  basic  elements  of  quality  assurance  policy:  the 
first  is  the  overt  policy  contained  in  Section  14  of  the  Defense  Acquisition 
Regulation  (DAR).  The  second  relates  to  product  quality  as  one  part  of  the 
overall  DoD  contracting  strategy.  The  overall  strategy  has  one  primary 
thrust,  profit. 

Purchasing  offices  are  instructed  to  increase  or  decrease  profit 
based  on  contractor  "performance, "  and  quality  and  reliability  are  identified 
as  elements  of  the  performance  evaluation.  From  the  DAR  evaluation,  it 
became  clear  that  profit  is  for  all  practical  purposes,  the  single  procurement 
motivational  device.  Various  contracting  techniques  principally  involving 
profit  incentives,  have  been  based  on  this  premise.  Reliability  policy 
results  in  equating  profit  to  performance  on  instant  contracts.  Quality 
assurance  policy  has  clearly  not  reflected  this  overall  procurement  assumption, 
although  there  is  direction  to  collect  quality  history  and  utilize  it  to 
eliminate  contractors  with  unsatisfactory  performance  history. 

The  DAR  assumption  that  profit  is  for  all  practical  purposes  the 
only  contractor  motivation  is  of  questionable  validity  on  two  basic  points: 

(l)  research  performed  on  this  project  and  by  other  investigators  indicates 
that. there. are. numerous  motivations  other  than  profit  for  contractor's 
participation  in  the  Defense  market  (these  motivations  relate  to  life  cycle 
phases  that  firms  experience  as  they  grow  and  expand),  and  (2)  profit  may  not 
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be  the  prime  objective  on  some  contracts,  and  contractor  negotiations 
relating  to  tradeoffs  may  be  based  on  entirely  different  goals  than  what 
the  government  negotiators  assumes  to  be  the  contractor's  goals. 

Despite  the  admonitions  in  the  DAR  and  other  policy  to  relate 
profit  to  performance  (quality),  there  are  no  institutionalized  procedures 
for  dcng  so.  In  the  past,  performance  factors  which  contained  quality 
and  delivery  data  were  deleted  from  the  Weighted  Guidelines  negotiating 
methodology  based  on  a  decision  that  no  objective  measure  had  been  developed 
and  use  of  the  factor  had  been  erratic.  Although  performance  history  is 
theoretically  still  retained  as  a  factor  in  source  selection,  its 
effectiveness  is  limited. 

A  second  profit  issue,  even  if  profit  is  utilized  to  reward 
contractors,  is  its  effectiveness  and  relationship  to  product  quality. 

Industry  does  continue  to  raise  the  profit  question,  but  any  direct  relation¬ 
ship  between  profit  and  quality  is  difficult  to  establish  and  the  investigation 
in  this  project  failed  to  find  evidence  to  support  the  contention  that  profit 
directly  affects  quality.  There  is  an  open  question  relating  as  to  how 
increased  profit  affects  personnel  who  are  in  a  position  to  influence  the 
quality  of  products.  The  findings  indicate  that  motivational  rewards  such 
as  management  recognition  of  good  workmanship  and  performance  are  more 
effective  than  financial  rewards. 

It  is  generally  accepted  that  the  Defense  market  is  not  a  classic 
market  in  the  ordinary  sense.  The  type  of  products,  the  highly  structured 
purchasing  system,  the  differing  risk  exposures  and  other  factors  all  impact 
on  the  normal  commercial  profit  incentive  considerations.  The  DAR  negotiating 
premise  that  profit  is  always  the  single  most  important  factors  is  a  gross 
oversimplification  of  a  complex  people-market  environment. 

Insofar  as  motivation  per  se  is  concerned,  the  procurement  policy 
is  essentially  silent  and  neither  encourages  nor  discourages  motivational 
programs.  Zero  Defects,  the  best  known  and  most  recommended  motivational 
program-  no  longer  has  DoD  recognition,  although  it  was  reported  by  inter¬ 
viewees  as  a  positive  program  with  benefits  which  reportedly  have  been 
continued  to  be  utilized  by  some  firms. 

No  evidence  was  found  in  the  study  which  indicated  that  the  USAF 
was  accepting  non-conforming  material.  In  addition,  most  contractors  reported 
that  they  perceive  their  product  quality  exceeds  requirements.  However,  there 
is  ample  confusion  surrounding  the  product  quality  question.  Product  quality 
is  a  function  first  of  adequate  definition  of  requirements,  secondly,  a  design 
to  fulfill  those  requirements,  and  lastly,  conformance  to  the  design.  The  USAF 
quality  assurance  program  concentrates  on  the  conformance  element  and  does  so 
by  requiring  the  firm  to  implement  a  product  quality  management  system.  Air 
Force  quality  assurance  efforts  then  are  primarily  spent  on  enforcing  the 
design  through  evaluation  of  the  management  system.  Definition  of  requirements 
and  excellence  of  design  are  functions  which  have  little  quality  assurance 
involvement,  although  it  is  clear  that  a  product  conforming  to  a  poor  design 
or  a  design  conceived  in  error,  will  result  in  a  poor  product  from  the  user’s 
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view.  Numerous  contractors  reflected  the  view  that  poor  statements  of 
requirements  constitute  a  serious  quality  problem.  It  is  probable  that 
many  quality  complaints  have  their  genesis  in  design  or  definition  of 
requirements  and  are  beyond  the  control  of  the  normal  USAF  quality 
assurance  sphere  of  influence. 

C.  CONCLUSIONS 

In  summary,  the  following  six  key  observations  relate  to  DoD 
procurement  (quality  and  reliability)  policy  as  they  affect  product/system 
quality: 


o  The  DAR  premise  is  that  profit  and  quality  are 
directly  relatable; 

o  Profit  should  be  related  to  performance,  past 
or  present; 

o  When  profit  is  determined  utilizing  the  Weighted 
Guideline  Method,  past  performance  is  not  a 
consideration; 

o  There  is  no  organized  method  for  developing  and 
providing  contractor  historical  quality  data  for 
evaluation  in  the  source  selection  process; 

o  There  is  essentially  no  DAR  policy  relating  to 
motivational  approaches  that  affects  junior  or 
non-supervi sory  contractor  personnel;  and 

o  Section  14  of  the  DAR  does  not  interface  with 
Sections  1  and  3  on  product  quality  matters. 

The  USAF  in-plant  quality  assurance  program  was  not  examined  in 
any  detail  in  this  investigation  as  it  fell  beyond  the  project  scope  and 
definition.  In  general  terms,  however,  the  in-plant  quality  assurance 
program  has  been  built  on  the  contractor  responsibility  concept  and  no 
major  conceptual' changes  have  been  made  for  a  considerable  number  of  years. 

The  program  is  assumed  to  be  effective  and  it  interfaces  well  with  other 
contract  administration  functions.  The  significance  of  the  acceptance  role 
is  a  principal  and  vital  element  of  the  procurement  function  which  does  set 
it  apart  from  routine  surveillance  functions.  There  are  other  approaches 
which  differ  from  the  current  USAF  quality  assurance  methodology  such  as  those 
used  by  commercial  air  carriers  which  must  also  be  assumed  to  be  effective. 
When  the  costs  of  the  large  USAF  in-plant  programs  are  considered,  there  may 
be  significant  differences  in  efficiency.  Whether  or  not  a  more  efficient 
approach  can  be  developed  for  USAF  procurements  is  an  open  question  that 
requires  a  complete  examination  to  resolve. 

D.  RECOMMENDATIONS 

Three  principal  recommendations  to  improve  USAF  contractor 
quality  and  reliability  policy  result  from  this  study: 
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o  That  the  proposed  contractor  performance  /eval  uati on 
summary  be  implemented,  thus  responding,  to  the  need 
for  an  Air  Force  corporate  quality  memory. 

o  That  the  proposed  Cost  Plus  Award  Fee  incentive 
approach  be  implemented  as  a  standard  contracting 
approach  designed  to  improve  quality. 

o  That  the  recommended  additions  and  revisions  to 
the  DAR  for  introducing  additional  motivational 
procedures  into  the  DAR  contracting  methodology 
be  adopted. 

These  recommendations  are  incorporated  in  Section  VI  of  this 
report.  Obviously,  any  program  of  the  magnitude  of  the  USAF  quality 
assurance  program  is  always  subject  to  small  incremental  improvements. 

These  recommendations  are  made,  however,  on  the  premise  that  if  followed, 
they  will  result  in  significant  improvements  in  the  quality  assurance 
function.  It  is  recommended  that  they  be  implemented  on  a  trial  basis 
and,  if  necessary,  improved  upon  and  then  made  an  integral  part  of  the 
USAF  quality  assurance  policy.  Other  observations  relating  to  the  quality 
program  may  be  seen  in  Appendix  VI-5. 

A  collateral  requirement  of  this  project  was  to  develop  a  recom¬ 
mended  Guidance  Document  entitled,  USAF  Acquisition  Managers  Quality  Guide, 
for  use  by  Air  Force  quality  and  reliability  assurance  managers,  contracting 
officers  and  acquisition  managers. 

The  guidance  document  was  developed  as  an  adjunct  to  existing 
policies  and  procedures.  It  was  designed  to  give  project  managers  an 
overview  of  how  product  quality  and  reli ability  policy  interrelate  with 
other  procurement  policy  and  to  illustrate  how  those  policies  could  be 
used  to  improve  product  quality. 

Specific  procedural  recommendations  were  made  to  improve  quality 
of  design  and  provide  additional  product  quality  data  for  use  in  system 
milestone  decisions.  The  Quality  Guide  is  provided  as  Volume  2  of  this 
report. 
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CONTRACT  INCENTIVE  FOR  QUALITY  ASSURANCE 
PART  I  -  APPROACH  AND  ANALYSIS 


A.  INTRODUCTION 

Appendix  A  presents  the  results  of  a  questionnaire  survey  relating 
to  quality  assurance,  reliability  and  motivational  interfaces.  It  was 
conducted  by  the  International  Technology  Corporation  (INTEC)  for  the  USAF 
Business  Research  Management  Center.  The  information  obtained  has  been 
used  as  part  of  the  research  for  the  investigation. 

The  objective  of  the  survey  was  to  assess  the  practices  and  per¬ 
ceptions  of  commercial  manufacturing  organizations  in  regard  to  motivational 
perceptions  and  product  quality  relationships.  The  respondents  were  primarily 
firms  that  were  involved  in  government  procurements.  A  distinct  effort  was 
made  to  include  other  firms  who  did  no,  or  at  least  a  limited  amount  of, 
government  business.  The  survey  was  buttressed  by  interviews  with  personnel 
from  industry  and  with  government  personnel,  primarily  but  not  exclusively 
USAF. 


1  .  Synopsis  of  Survey  Results 

The  survey  was  intended  primarily  to  provide  insight  into  moti¬ 
vational  perceptions  of  the  industrial  element  of  the  purchaser/supplier 
relationships.  At  the  same  time  the  opportunity  was  taken  to  examine  other 
product  quality  considerations  such  as  how  is  quality  specified  and  how  is 
it  measured  and  what  techniques  are  used  in  the  industrial  environment  to 
encourage  the  delivery  of  conforming  material.  Respondents  were  given  the 
opportunity  to  provide  their  recommendations  as  to  how  USAF  product  quality 
could  be  improved. 

In  general,  respondents  reported  that  current  USAF  quality  manage¬ 
ment  systems  are  good  although  other  systems  are  also  successful,  e.g., 
commercial  airline  techniques.  The  respondents  basically  reported  that  USAF 
motivational  contracting  devices  seem  to  be  beneficial  although  many  firms 
have  not  experienced  them.  The  questionnaire  provided  insight  into  how  the 
industrial  community  views  the  relationship  between  USAF  contracting 
methodology  and  product  quality. 

2 .  Organization  of  the  Report 

Part  I  of  this  Appendix  provides  a  complete  documentation  of  the 
survey  findi ngs  with  all  results  presented.  This  approach  is  taken  to  minimize 
any  interpretative  bias.  Conclusions  drawn  are  substantiated  by  questionnaire 
data. 
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Section  B  provides  a  discussion  of  methodology  used  in  developing 
the  questionnaire.  Section  C  presents  the  research  results  on  a  question 
by  question  basis.  Section  D  contains  respondent  information  and  findings. 

Part'll  of  this  Appendix  provides  the  transmittal  letter  and  sample 
of  the  questionnaire  employed  and  Part  III  lists  the  respondents  who  participated 
in  the  survey  and  chose  not  to  remain  anonymous. 

B.  METHODOLOGY 

1 .  Approach 

The  questionnaire  was  designed  to  provide  information  that  could 
not  be  reasonably  obtained  from  personal  interviews  alone  or  extensive 
literature  searches.  It  supplements  research  efforts  and  studies  in  the 
same  genera  field  and  was  intended  to  elicit  information  from  sources  not 
heretofore  approached  in  USAF  or  DoD  contracting  studies.  The  questionnaire 
survey  approach  permitted  information  to  be  collected  from  a  variety  of 
industrial  managers  that  could  not  have  been  accomplished  otherwise. 

2 .  Questionnaire  Design 

A  significant  effort  was  devoted  to  development  of  a  questionnaire 
which  would  provide  the  maximum  information  from  the  broadest  possible  base 
with  as  little  time  and  effort  as  possible  on  the  part  of  the  recipient. 

Since  the  questionnaire  would  need  to  be  complete  in  itself,  it  had  to  be 
as  comprehensive  as  possible  and  sensitive  to  attitudinal  postures  within  the 
manufacturing  community.  It  had  to  be  adaptable  to  a  complete  spectrum  of 
products  and  industries  and  to  provide  insight  into  the  respondents  that 
would  allow  an  accurate  interpretation  of  results.  As  a  final  test  of  the 
format,  the  questionnaires  were  provided  to  members  of  a  local  chapter  of 
the  American  Society  of  Quality  Control  as  a  controlled  test.  The  question¬ 
naire  was  then  refined  and  forwarded  to  all  recipients. 

The  list  of  organizations  to  whom  the  survey  would  be  provided  was 
developed  from  two  sources.  The  first  was  from  industrial  registers  such  as 
Standard  and  Poors  which  was  intended  to  assure  that  a  large  group  of  respondents 
could  be  developed.  Secondly,  recipients  were  identified  with  the  assistance 
of  officials  of  the  USAF  and  of  related  industrial  associations  such  as  the 
Institute  of  Electrical  and  Electronic  Engineers  (Reliability  Society), 

American  Society  for  Quality  Control  and  the  Society  of  Logistics  Engineers. 

A  total  of  two  hundred  and  forty  five  questionnaires  were  distributed  in 
December  1979  and  January  1980,  from  which  eighty  two  responses  were  received; 
a  33  percent  return. 

The  questionnaire  was  designed  with  the  option  of  utilizing 
electronic  data  processing  evaluation  procedures.  It  was  later  decided, 
however,  because  a  large  portion  of  the  respondents  chose  to  elaborate  on 
their  responses,  that  the  subjective  evaluations  possible  from  manual 
tabulations  might  provide  a  more  meaningful  analysis.  The  manual  tabulation 
coupled  with  the  relative  ease  of  the  material  and  presentation  methodology,’ 
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proved  to  be  sati sfactory .  Analysis  is  provided  either  by  assigning  per¬ 
centages  based  on  the  total  number  of  respondents  to  questions  and  by 
summarizing  innovative  and  open  ended  questions.  In  some  cases,  particular 
answers  were  presented  in  total  when  some  insight  or  response  appeared 
particularly  meaningful,  and  would  not  bias  the  results.  In  some  cases, 
no  comments  were  provided  since  the  data  results  were  self-evident. 

3 .  Personal , Interviews 

i he  objectives  of  the  personal  interviews  were  identical  with 
those  of  the  questionnaire.  The  intent  was  to  supplement  the  impersonal 
questions  with  questions  and  answers  in  a  dynamic  interview  environment. 

The  interviews  were  to  learn  more  of  the  general  philosophies  and  beliefs 

of  the  interviewees  as  to  how  they  viewed  procurement  problems  and  motivational 
considerations . 

No  formal  agenda  was  used  and  the  questions  were  open  ended  in 
order  to  encourage  wide  open  thought  processes  and  discussions  in  contrast 
to  simple  yes  or  no  discussion  limiting  questions.  Several  government 
personnel  were  interviewed  at  the  AFSC  division  offices  and  at  the  manu¬ 
facturers  facilities.  In  all  cases,  interviewees  and  questionnaire 
respondents  were  assured  of  the  confidentiality  of  their  discussion. 

Selected  interview  results  are  included,  but  not  in  a  statistical 
eval uation . 

C.  QUESTIONNAIRE  FINDINGS 

The  findings  that  follow  on  the  next  pages  represent  the  data 
contained  in  the  32  questionnaires  that  were  returned  out  of  the  245  mailed. 
Eighty  two  is  a  substantial  base  for  statistical  purposes  and  the  responses 
do  provide  a  meaningful  insight  into  the  current  state  of  mind  of  industry 
personnel.  Unless  otherwise  noted,  the  numbers  shown  represent  the  percentage 
of  responses  to  each  question  and  "no  responses"  are  included  as  part  of  the 
base.  Due  to  fractional  carryovers,  totals  may  vary  around  100%  slightly. 

In  some  cases,  analysis  may  be  improved  by  assigning  weights  or  factors,  and 
this  possibility  is  discussed  in  some  questions  and  one  suggested  method  is 
provided.  It  is 'believed,  however,  that  it  is  essential  to  provide  raw  data 
without  analysis  to  permit  other  analysts  to  adjust  the  data  response  in 
whatever  fashion  may  be  desired. 

Many  of  these  questions  have  been  discussed  before  in  other  USAF 
surveys;  this  questionnaire  was  designed  to  provide  a  useful,  up-to-date 
and  valid  method  of  integrated  analysis.  It  is  intended  to  examine  motivational 
perception  in  industry  and  to  provide  a  unified  and  contemporary  industry 
view  of  quality  associated  contracting  methodology.  It  also  provides  insight 
into  quality  management  techniques  employed  by  industry  and  views  on  specifying 
and  measuring  product  quality. 

Analysis  of  the  data  shows  that  industry  is  confident  it  is 
supplying  equipment  meeting  or  exceeding  requirements.  It  shows  a  diversity 
of  opinions  about  motivational  practices,  although  psychological  incentives 
are  seen  as  a  strong  motivator,  and  that  profit  is  not  a  great  concern  to 
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operators,  or  hands-on  employees,  as  it  effects  quality.  Quality  programs 
are  seen  as  highly  beneficial  for  product  quality  and  inadequate  technical 
descriptions  are  seen  as  a  major  problem. 

While  the  summary  could  be  further  expanded  based  on  other  areas 
examined  in  the  questionnaire,  it  is  sufficient  to  say  that  there  are 
significant  opinions  reported  by  respondents  in  other  areas  related  to 
product  quality  and  contracting  methodology.  Summary  analysis  has  been 
provided  on  a  question  by  question  basis  and  further  examination  of 
quality  issues  will  require  a  review  of  specific  points  raised  singularly 
in  the  questionnaire. 


QUESTIONNAIRE  RESULTS 


1.  Please  describe  the  type  of  products  your  firm  is  currently 
designing, .developing,  manufacturing  or  maintaining  for  your 


primary  customers. 


USAF 

(1) 

El ectronic 

54% 

(2) 

Air  Frame 

9% 

(3) 

Mechanical 

21% 

(4) 

Hydraul ic 

10% 

(5) 

Munitions 

6% 

(6) 

Pro pul  si  on 

5% 

OTHER 

DoD 

OTHER 

GOVERNMENT 

COMMERCIAL 

in 

13^ 

in 

* 

48% 

7% 

6% 

10% 

24% 

22% 

18% 

15% 

11% 

15% 

9% 

6% 

2% 

5% 

5% 

5% 

COMMENT: 

Quest-ions  t  through  6  give  a  view  of  the  customers  and  the  types  of 
products  produced  by  contractors  responding  to  the  questionnaire.  The 
opportunity  to  identify  other  products ,  (questions  7,  8,  and  9 )3 
identified  a  series  of  miscellaneous  products  of  such  a  diversity  that 
no  significance  could  be  attached  to  them. 


2.  Please  describe  your  Quality  Assurance  and  Reliability  Program. 


(10) 

MIL-Q-9858A 

70% 

(ID 

MIL-I-45208A 

50% 

(12) 

NASA  NHB  5300.4  Series 

34% 

(13) 

FAA-STD-01 3A 

16% 

(14) 

MIL-STD-785 

40% 

(15) 

MIL-STD-470 

34% 

(16) 

Commercial  Standard  (please  identify) 

21% 

(17) 

Other  (Note:  Some  contractors  may 

qualify  under  several  systems) 

21% 

A- 5 


3.  Does  your  firm  normally  utilize  a  design  review  program?  Please 
check  the  appropriate  answer. 


(18) 

YES  90% 

NO 

6% 

NO  RES 

P0NSE 

4% 

4. 

If  the  answer  above  is 

yes,  please  indicate  which  functional 

groups 

or  disciplines  normally 

participate  in 

the  review. 

YES 

NO 

OFTEN 

SELDOM 

NO 

RESPONSE 

(19) 

Engineering 

67% 

0% 

21% 

1% 

11% 

(20) 

Rel iabil ity 

48% 

1% 

24% 

5% 

22% 

(21) 

Maintainabil ity 

21% 

5% 

29% 

12% 

33% 

(22) 

Quality  Assurance 

40% 

1% 

34% 

7% 

18% 

(23) 

Manufacturing 

30% 

1% 

35% 

12% 

22% 

(24) 

Your  Project  Manager 

48% 

2% 

28% 

4% 

18% 

(25) 

Customer's  Project 
Manager 

15% 

13% 

23% 

22% 

26% 

5. 

Does  your  company  utilize  the  handbooks  or  other  standard  documents 
which  provide  the  type  of  guidance  shown  below? 

(26) 

Minimum  quality  levels 

to  meet  selected  performance  levels 

YES  33% 

NO 

13% 

NO  RESPONSE 

4% 

(27) 

Minimum  costs  associated  with  selected  quality 

1  evel s 

YES  26% 

NO 

66% 

NO  RESPONSE 

8% 

COMMENT: 

These  questions  (26  and  27)  provide  insight  into  the  extent  to 
which  methodology  is  available  to  relate  design  to  quality  and 
costs  to  quality. 


A-6 


6. 


If  your  firm  has  had  experience  with  anv  nf  j-ho 

approaches  please  indicate  the  impact  you  wo^d  elpecrfhem"^^^1 
on  materiel  quality.  7  wuuia  expect  them  to  have 


NEGATIVE 

IMPACT 

(28)  Correction  of  deficiencies 

0% 

(29)  Reliability  Improvement 
Warranty 

1% 

(30)  Life  Cycle  Cost  Incentive 

0% 

(31)  Value  Engineering 

6% 

(32)  Fail u re- Free  Warranty 

0% 

\33)  Quality  Assurance  Program 

0 % 

(34)  Reliability  Program 

0% 

(35)  Maintainability  Program 

1% 

(36)  Liquidated  Damages 

6% 

(37)  Design  to  Unit  Product  Cost 

12% 

(38)  Zero  Defects 

4% 

^39)  Multi-level  Acceptance 
Sampling  Plans 

2% 

(40)  Reliability/Quality  Incentive  0% 

(41)  Performance  Incentive 

2% 

(42)  Reliability  Demonstration 

2% 

(43)  Draft  Request1 for  Proposal 

1% 

(44)  Award  Fee 

0% 

NO  SLIGHTLY 

IMPACT  BENEFICIAL 

VERY  NO 

beneficial  RESPONSE 

18% 

17% 

48%  . 

17% 

11% 

19% 

16% 

53% 

12% 

26% 

12% 

50% 

14% 

43% 

14% 

23% 

12% 

11% 

18% 

59% 

4% 

13% 

76% 

7% 

1% 

21% 

61% 

17% 

5% 

27% 

38% 

29% 

24% 

9% 

0% 

61% 

17% 

15% 

7% 

49% 

18% 

23% 

11% 

44% 

17% 

28% 

13% 

40% 

5% 

20% 

28% 

47% 

10% 

17% 

32% 

39% 

10% 

27% 

36% 

25% 

12% 

21% 

15% 

51% 

11% 

18% 

16% 

55% 

COMMENT: 


Winiln'T/the  ToTffftiZfZeZ  as  «»  respondents 

quality.  The  HO  RESPONSE  eolurm  pfZiZfZnsZZtZ^  t°Tlvd  prosing 
these  concepts  in  contracts  Nn  n-hhJ?  „ght  into  the  usage  of 

identified.  ’  **  PX’O&PGms  of  consequence  were 
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7. 


Would/does  the  inclusion  in  a  contract  of  any  of  the  contractual 
approaches  listed  in  question  6  cause  the  Quality  Assurance 
activities  in  your  firm  to  staff  or  plan  actions  in  other  than 
the  customary  manner? 

(45)  '  YES  48%  NO  46%  NO  RESPONSE  16% 

COMMENT: 

Every  program  listed  in  question  6  was  identified  at  least  once  as 
causing  a  change  in  staffing.  Those  most  often  identified  were  29, 

30,  32,  40  and  42.  The  CAP  program  is  also  identified  as  one  which 
would  result  in  changing  staffing  levels. 

8.  Please  identify  those  of  the  following  factors  which  have  a  degrading 
impact  on  the  quality  of  materiel  you  produce. 

NO  SLIGHT  SIGNIFICANT  SERIOUS  NO 

DEGRADATION  DEGRADATION  DEGRADATION  DEGRADATION  RESPONSE 


(46) 

Rush  Job/Schedule 

8% 

39% 

34-% 

6% 

13% 

(47) 

Subcontractors 

30% 

37% 

14% 

'  6% 

13% 

(48) 

Poor  Working  Conditions 

12% 

29% 

25% 

6% 

28% 

(49) 

Poor  Specifications 

7% 

15% 

30% 

23% 

25% 

(50) 

Poor  Morale 

11% 

23% 

30% 

13% 

23% 

(51) 

Job  Insecurity 

14% 

32% 

23% 

4% 

27% 

(52) 

Lack  of  Pride  in  Work 

8% 

22% 

37% 

10% 

23% 

(53) 

Lack  of  Skills 

6% 

18% 

31% 

23% 

22% 

COMMENT: 

These  figures  provide  insight  into  recognizable  problems 
to  poor  quality.  Poor  specifications  and  lack  of  skills 
significant  problems. 

which  contribute 
are  clearly 

9. 

Do  you  believe  it  is  realistic 
to  be  achieved? 

to  include 

in  a  contract 

a  qua! ity  level 

(55) 

YES  83% 

NO 

15% 

NO  RESPONSE  2% 
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10.  If  the  answer  to  9  is  yes,  what  term  of  specificity  would  you  recommend? 


(56)  Performance  Specification  35% 

(57)  Market  Grade  4% 

(58)  Design  Specification  24% 

(59)  Brand  Name  4% 

(60)  Catalog  Description  5% 

(61 )  Part  Number  7% 


(62)  Acceptable/Unacceptable  Defect  Levels  21  % _ 

COMMENT: 

The  "specification  .of  quality "  and  "duality  levels "  are  frequently  used 
terms.  This  series  of  questions  is  designed  to  show  what  these  terms 
mean  and  how  quality  is  specified  in  practice. 

11.  Would  you  describe  the  materiel  that  your  firm  builds  as  exceeding 
customer  specified  quality  requi rements? 

(64)  YES  85%  NO  12% _  NO  RESPONSE  3% _ 

COMMENT.: 

Allegations  of  "poor  quality"  are  frequently  made.  These  answers  clearly 
indicated  that  most  contractors  beiieve  they  exceed  requirements .  The 

NO  must  be  viewed  not  as  to  meeting  requirements  but  as  exceeding  them 
One  respondent  answered  as  follows ,  YES,  "forced  by  in-plant  AFPRO. " 

12.  Do  you  feel  that  the  inclusion  of  life  cycle  cost  type  of  incentives,  e.g.. 
Reliability  Improvement  Warranty  or  fixed  price  maintenance  options  in 
design/development  contracts,  would/does  result  in  measurable  improvement 
in  the  quality  of  materiel  normally  produced  by  your  firm? 

(65)  YES  •  39%  NO  52%  NO  RESPONSE  9% _ 

13.  If  you  were  to  assign  responsibilities  for  product  defects  which  occur  in 
your  plant,  what  proportions  of  that  responsibility  would  you  assign  to 
each  of  the  following  categories  of  personnel? 


(66)  Corporate  or  Top  Level  Management  31% 

(67)  Middle  Level  Management  11% 

(68)  First  Level  Supervisor  14% 

(69)  Operators  or  Non-Supervi sory  25% 

(70)  Design  Engineers  19% 


COMMENT: 


This  question  was  intended  for  correlation  with  normal  motivational 
contracting  procedures .  It  affirms  tnat  all  levels  of  the  organization 
share  responsibility  for  product  defects.  The  operator, /non- supervisory 
category  does  not  differ  significantly  from  the  20%  generally  assumed 
to  be  controllable  by  that  group. 

14.  How  do  you  rate  the  following  management  techniques  or  methods  to  assure 
that  your  firm  is  obtaining  materiel  from  suppliers  that  conforms  to 
requi rements . 


NO  SLIGHT 

BENEFIT  BENEFIT 

SIGNIFICANT 

BENEFIT 

BEST 

METHOD 

NO 

RESPONSE 

(71)  Your  Inspectors  in 

4% 

29% 

39% 

25% 

3% 

Supplier's  Plant 

(72)  Receiving  Inspection 

0 % 

12% 

56% 

32% 

0% 

(73)  Supplier  Surveys 

0% 

44% 

46% 

7% 

3% 

(74)  Clear  Definition  of 

0% 

0% 

46% 

54% 

0% 

Requirements 

(75)  Financial  Incentives 

17% 

46% 

27% 

4% 

6% 

(76)  Psychological  Incentives 

23% 

48% 

16% 

1% 

12% 

COMMENT: 


It  is  significant  to  note  that  "clear  definition  of  requirements"  stands 
out  as  a  mag  or  concern  in  addition  to  the  expected  techniques  such  as 
receiving  inspection  and  profit .  See  question  49. 

15.  How  do  you  rank  the  following  factors  in  terms  of  a  problem  causing  the 
production  of  non-conforming  materiel? 

NO  SLIGHT  SIGNIFICANT  SERIOUS  NO 

PROBLEM  PROBLEM  PROBLEM  PROBLEM  RESPONSE 


(77) 

Poor  Working  -Conditions 

14% 

51% 

18% 

7% 

10% 

(78) 

Low  Pay 

31% 

42% 

13% 

2% 

12% 

(79) 

Poor  Instructions 

0% 

7% 

37% 

47% 

9% 

(80) 

Lack  of  Motivation 

3% 

25% 

45% 

15% 

12% 

(81) 

Lack  of  Interest 

7% 

35% 

39% 

13% 

6% 

(82) 

Tight  Schedules 

2% 

36% 

42% 

12% 

8% 

(83) 

Low  Profit 

39% 

34% 

18% 

2% 

7% 

(84) 

Subcontractors 

12% 

53% 

19% 

9% 

7% 

(85) 

Poor  Specification 
of  Requirements 

4% 

8% 

43% 

43% 

2% 
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COMMENT: 


This  is  a  redirection  of  earlier  questions  aimed  toward  the  causes 
of  non-conforming  materiel .  It  does  not  ask  the  respondent  about  his 
plant j  but' rather  his  own  opinion  in  a  theoretical  sense. 

What  do  you  believe  is  the  single  most  effective  motivational  device 
that  can  be  incorporated  into  a  contract/subcontract? 

CLEAR  PSYCHOLOGICAL  TEST 

INCENTIVES  REQUIREMENTS  PROFIT  PENALTIES  INCENTIVES  PROGRAM 

35%  24%  16%  10%  10%  3% 

Do  you  have  a  specific  method  or  procedure  for  determining  that  a 
contractor's/subcontractor's  products  are  such  poor  quality  that  no 
more  contracts  will  be  awarded? 

YES  81%  NO  1%  BASIS  IS  VARIABLE  0% 

If  your  answer  is  YES,  please  describe  your  method  briefly? 

COMMENT: 

This  question  relates  to  the  DoDD  4155.1  admonition  to  eliminate  contractors 
with  poor  quality  history ,  Eight  four  percent  of  the  respondents 
rely,  on  vendor  rating  systems.  It  is  assumed  that  the  contractors 
rating  system  includes  a  central  file  with  subcontractors  identified 
with  their  quality  history. 

Responses,  to  this  question  generally  fell  into  the  categories  shown 
in  Question  86,  Incentives  are  viewed  in  this  case  as  profit  that 
will  be  available  based  on  performance  as  opposed  to  the  singular 
profit  entry. 


QUALITY 

PROGRAM 

2% 


* 


18.  If  the  following  factors  are  considered  to  be  measures  of  quality, 
either  direct  or  indirect,  please  rate  them  in  one  of  the  indicated 
categories  by  circling  the  appropriate  number. 

NO  POOR  GOOD  BEST 
MEASURE  MEASURE  MEASURE  MEASURE 


(88) 

Engineering  Change.s 

10 

32 

35 

1 

(89) 

Defect  Levels  at  Acceptance 

Decision  Points 

1 

4 

53 

18 

(90) 

In-process  defect  rates 

1 

6 

54 

14 

(91) 

Compliance  with  delivery  schedules 

31 

34 

14 

0 

(92) 

Variations  from  life  cycle 
cost  projections 

21 

39 

10 

2 

(93) 

Defects  found  at  customer 
delivery  points 

0 

9 

52 

18 

(94) 

Customer  complaints  during 
usage 

2 

13 

44 

19 

(95) 

Process  control  effectiveness 

3 

9 

46 

13 

(96) 

Product  qualification 

4 

19 

40 

10 

(97) 

Performance  Testing 

0 

6 

51 

23 

(98) 

Receiving  Inspection  Results 

1 

19 

54 

8 

(99) 

Audit  Findings 

3 

18 

50 

5 

(100) 

Scrap  and  Rework  Cost 

3 

14 

48 

10 

(101) 

Materiel  Review  Board  Actions 

3 

31 

40 

3 

(102) 

Warranty  Costs 

5 

17 

38 

15 

(103) 

Field  Maintenance  Costs 

7 

17 

30 

14 

COMMENT: 

This  question  relates  to  the  "how  to 

measure 

quality " 

issue. 

It  is 

^  ^  ^  J mu  measure  =  u3  roar  Measure  = 

f°°d  Measure  =  3,  and  Best  Measure  =  4 ,  to  rank  the  relative  score  for 
eacn  item.  If  this  is  done,  it  will  show  scores  closely  grouped  with  the 
exception  of  91  and  92  which  rank  lower.  Question  104  received  almost  no 
response  except  for  feedback  which  was  identified  6  times .  These  answers 
are  not  tn  percentages . 
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19. 


If  your  firm  has  been  provided  Government  furnished  materiel,  i.e., 
component,  subsystem,  etc.,  how  do  you  perceive  its  impact  on  the 
quality  of  the  delivered  end  item,  as  compared  to  end  item  quality 
where  you  provide  all  materiel. 


(105) 

Improved 

3% 

(106) 

Degraded 

43% 

(107) 

No  Difference 

46% 

No  Response 

8% 

20. 

Does  your  company  have  an 

employee  motivation  program? 

008) 

YES  67% 

NO  28%  NO  RESPONSE 

5% 

21 

How  do  you  rate  the  following  management 
ability  to  motivate  employees  to  strive 
levels? 

practices  in  terms  of  their 
for  high  quality  workmanship 

NO 

BENEFIT 

SLIGHT 

BENEFIT 

SIGNIFICANT 

BENEFIT 

3EST 

METHOD 

NO 

RESPONSE 

(109) 

Increase  Pay 

15% 

50% 

25% 

4% 

6% 

(no) 

Official  Recognition 

0% 

4% 

54% 

39% 

3% 

(111) 

Improved  Working 
Conditions 

4% 

41% 

43% 

3% 

9% 

(112) 

Varied  Duties 

6% 

56% 

28% 

4% 

6% 

(113) 

Suggestion  Programs 

8% 

55% 

26% 

1% 

9% 

(114) 

Long-Term  Contracts 

17% 

29% 

28% 

7% 

19% 

(115) 

Special  Bonus  or  Other 
Cash  Awards 

12% 

35% 

37% 

10% 

6% 

COMMENT: 

This  question  vs  aimed,  at  identifying  indLixstrial  practices  to  motivate 
operator  category  employees.  Respondents  were  asked  to  identify  other 
areas ,  no  significant  responses  were  obtained.  They  may  also  be  ranked 
by  using  weighting  factors  as  discussed  in  Question  18.  If  so  ranked , 
official  recognition  will  rank  svgnificantly  above  bonuses  or  other 
cash  awards. 
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22.  Does  the  accounting  system  in  your 
the  following  categories  of  qua!  it; 


YES 


(117) 

Defect  Prevention 

51  % 

(118) 

Defect  Detection 

65% 

(119) 

Defect  Correction  (Including 

Warranty  Costs) 

63% 

COMMENT: 

firm  accurately  account  for  all  of 
costs? 


NO 

NO  RESPONSE 

44% 

5% 

29% 

6% 

32% 

5% 

The  answers  to  this  question  show  that  quality  cost  accounting  remains 
as  an  open  quality  management  problem. 


Please  identify  the- emphasis  government  contract  managers  place  on 
the  following  contract  objectives  by  circling  the  appropriate  number. 


LOW 

EMPHASIS 

MODERATE 

EMPHASIS 

HIGH 

EMPHASIS 

NO 

RESPONSE 

(120) 

Schedul e 

4% 

12% 

76% 

8% 

(121) 

Cost 

4% 

16% 

68% 

12% 

(122) 

Qual i ty  of  Materi el 

23% 

35% 

34% 

8% 

24. 

What  are  your  recommendations  for 
of  the  U.S.  Air  Force  Materiel? 

improving  the  quality  and 

rel iabil i ty 

COMMENT: 

Many  respondents  took  this  opportunity  to  provide  recommendations  and  a 
total  of  10’S  suggestions  were  received.  Many  were  thorough  and  significant. 
There  was  a  wide  range  of  suggestions  and  the  primary  recommendations  were , 
in  no  particular  order ,  the  following:  (l)  improve  contracts,  (2)  improve 
technical-  descriptions ,  i  3)  improve  contract  administration  procedures 
and  upgrade  personnel,  (4)  enforce  contracts  uniformly,  (5)  reduce  paperwork, 
and  (o)  consider  past  performance  in  source  selection.  One  respondent 
after  listing  his  suggestions ,  added  "The  first  time  a  contract  is  awarded 
because  it  is  a  quality  house,  then  we  can  stop  asking  this  question.  " 

The  fact  that  these  recommendations  have  been  mads  previously  does  not 
detract  from  their  validity.  In  particular,  the  "improve  technical 
descriptions"  appear  elsewhere  in  the  report  as  a  significant  problem. 
Inadequate  product  description  problems  cannot  be  overcome  with  quality 
management  techniques.  The  multiplicity  of  answers  also  indicates  the 
problems  of  using  uniform  contracting  procedures  despite  the  different 
situations  extant  in  the  various  management  modes  of  different  firms. 
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D.  RESPONDENT  INFORMATION  AND  FINDINGS 

The  preponderance  of  the  respondents  were  professionally  associated 
with  quality  assurance,  55  percent,  or  reliability,  19  percent.  Forty  six  per¬ 
cent  of  them  were'  in  corporate  or  top  level  management  positions  with  50  percent 
in  middle  management.  Seventy  seven  percent  of  the  firms  were  aerospace  product 
related  producers.  The  USAF  is  a  major  customer  for  48  percent  of  the  firms 
and  a  minor  customer  for- 32  percent.  Ninety  two  percent  of  the  firms  have 
been  in  existence  for  over  20  years.  Thirty  three  percent  have  experienced 
contracts  over  $50  million  and  52  percent  over  $1  million  in  the  last  five 
years.  They  are  principally  manufacturing  firms,  although  a  significant 
number  have  been  involved  in  research  activities. 

The  data  obtained  through  the  questionnaire  contains  several  biases. 
The  middle/upper  level  management  positions  associated  largely  with  quality 
assurance  and  reliability  responsibilities  can  be  assumed  to  have  influenced 
some  of  the  data.  This  is  not  necessarily  deleterious,  since  their  responsi¬ 
bilities  are  compatible  with  the  study  area  of  interest.  DoD  policy  is  not 
unfamiliar  to  these  individuals  and  its  impact  is  clear  to  them.  From  a 
motivational  standpoint,  the  study  may  not  accurately  reflect  the  perceptions 
of  the  operators  and  lower  level  management  personnel.  In  view  of  the  product 
defects  controllable  by  this  group,  it  would  be  desirable  to  conduct  a  second 
survey  aimed  specifically  at  these  individuals.  Overall,  the  questionnaire 
does  provide  significant  insight  into  DoD  policy,  product  quality  and 
reliability,  and  motivational  relationships. 
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PART  II 


SURVEY  QUESTIONNAIRE 


This  section  contains  the  transmittal 
copy  of  the  questionnaire  employed  in 


letter  and  a 
the  study. 
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A. 


TRANSMITTAL  LETTER 


/ 


International  Technology  Corporation 


Dear  Si r: 

At  the  request  of  the  United  States  Air  Force  we  have  undertaken  an 
investigation  into  defining  the  factors  in  contracting  procedures  that 
impact  on  the  quality  of  material  purchased  by  the  Air  Force.  We  are 
further  examining  current  contracting  procedures  to  determine  if  improve¬ 
ments  can  be  made  which  will  improve  the  quality  of  Air  Force  material. 

At  the  conclusion  of  our  investigation  we  will  provide  our  findings 
and  recommendations  to  the  United  States  Air  Force  for  their  consideration. 
It  is  possible  that  our  conclusions  will  ultimately  result  in  positive 
changes  to  current  contracting  procedures.  We  will,  therefore,  be  most 
appreciative  of  your  assistance.  Obviously,  the  greater  the  spectrum  of 
involvement  by  knowledgeable  people,  the  greater  the  validity  that  can  be 
attached  to  our  findings.  Your  opinions  will  be  included  in  our  sample 
and  will  be  treated  confidentially. 

We  sincerely  hope  that  you  will  find  the  necessary  time  to  complete 
the  questionnaire  and  return  it  in  the  enclosed  self-addressed  envelope. 

Sincerely, 


L.  R.  Webster 
President 


Enclosure 
LRW: jac 
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1670  Highway  A1  A,  Satellite  Beach,  Florida  32937,  305/777-1400 


SURVEY  ON  QUALITY  AND 

RELIABILITY-RELATED  CONTRACTUAL  INCENTIVES 


This ^ short  questionnaire  has  been  prepared  as  a  part  of  the  U  S 
Air  Force's  continuing  investigation  into  methods  of  improving  the’ 
quality  of  material  and  associated  contractual  incentive  procedures, 
■his  is  a  research  effort.  The  term  Quality  is  used  in  its  broadest 
sense  and  is  not  limited  to  conformance  to  design. 

This  questionnaire  provides  a  forum  for  you  and  your  organization 
to  offer  specific  aovice  to  the  Air  Force  as  to  what  can  be  done  to 
improve  contracting  procedures. 

Your-  individual  company  responses  will  be  confidential.  A  stamped 
self-addressed  envelope  is  provided  for  your  convenience. 

Thank  you  very  much  for  your  assistance. 


A-l  8 


B.  QUESTIONNAIRE 


CONTRACT  INCENTIVES  FOR  PROOUCT  QUALITY 


Please  describe  the  type  of  products  your  firm  is  currently  designing, 
developing,  manufacturing  or  maintaining  for  your  primary  customers  by 
circling  the  appropriate  number  below.  For  example, if  you  manufacture  for 
the  USAF, circle  1  in  Row  1.  This  same  method  will  be  employed  in  later 
questions , 


OTHER 

U.S.A.F.  DQD 


( 1) 

Electronic 

1 

2 

(  2) 

Ai r  Frame 

1 

2 

(  3) 

Mechanical 

1 

2 

(  4) 

Hydraulic 

1 

2 

(  5) 

Munitions 

1 

2 

•  (  6) 

Propulsion 

1 

2 

Other  Products  (Please  Specify) 

(  7) 

1 

2 

(  8) 

1 

2 

(  9) 

1 

2 

Please  describe  your  Quality  Assurance 
by  checking  the  appropriate  program  below. 

and  Rel 

(10) 

MIL-Q-9B58A 

(ID 

MIL-I-45208A 

(12) 

NASA  NHB  5300.4  Series 

(13) 

FAA  -  STD-01 3A 

(14) 

MIL-STD-785 

(15) 

MIL-STD-470 

(16) 

Commercial  Standard 

(Please  identify) 

(17) 

Other  (Please  identify) 

OTHER 

GOVT.  COMMERCIAL 


3 

3 

3 

3 

3 

3 


4 

4 

4 

4 

4 

4 


3 

3 

3 


4 

4 

4 


iability  organization 
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3. 


Does  your  firm  normally  utilize  a  design  review  program?  Please 
check  the  appropriate  answer. 

(18)  YES _  NO 


4*  If  .the  answer  above  is  yes,  please  indicate  by  circling  the  appropriate 

number  which  functional  groups  or  disciplines  normally  participate  in  the 
review. 


YES 

NO 

OFTEN 

SELDOM 

(19) 

Engineering 

1 

2 

3 

4 

(20) 

Rel iabil ity 

1 

2 

3 

4 

(21) 

Maintainability 

1 

2 

3 

4 

(22) 

Quality  Assurance 

1 

2 

3 

4 

(23) 

Manufacturing 

1 

2 

3 

4 

(24) 

Your  Project  Manager 

1 

2 

3 

4 

(25) 

Customer's  Project  Manager 

1 

2 

3 

4 

Does  your  company  utilize  handbooks  or  other  standard  documents  which 
provide  the  type  of  guidance  shown  below? 

(26)  Minimum  quality  levels  to  meet  selected  performance  levels 

YES  _  NO  _ 

(27)  Minimum  costs  associated  with  selected  quality  levels 

YES  : _  NO 
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6.  If  your  firm  has  had  experience  with  any  of  the  following  contractual 

approaches,  please  indicate  the  impact  you  would  expect  them  to  have  on  materiel 
quality  by  circling  the  appropriate  number. 

NEGATIVE  NO  SLIGHTLY  VERY 
IMPACT  IMPACT  BENEFICIAL  BENEFICIAL 


(28) 

Correction  of  Deficiencies 

1 

2 

3 

4 

(29) 

Reliability  Improvement 
Warranty 

1  - 

2 

3 

4 

(30) 

Life  Cycle  Cost  Incentive 

1 

2 

3 

4 

(31) 

Value  Engineering 

1 

2 

3 

4 

(32) 

Failure-Free  Warranty 

1 

2 

3 

4 

(33) 

Quality  Assurance  Program 

1 

2 

3 

4 

(34) 

Reliability  Program 

1 

2 

3 

4 

(35) 

Mai ntai nabi 1 i ty'  Program 

1 

2 

3 

4 

(36) 

Liquidated  Damages 

1 

2 

3 

4 

(37) 

Design  to  Unit  Production  Cost 

1 

2 

3 

4 

(38) 

Zero  Defects 

1 

2 

3 

4 

(39) 

Multi-level  Acceptance 

Sampling  Plans 

1 

2 

3 

4 

(40) 

Reliability/Quality  Incentive 

1 

2 

3 

4 

(41) 

Performance  Incentive 

1 

2 

3 

4 

(42) 

Reliability  Demonstration 

1 

2 

3 

4 

(43) 

Draft  Request  for  Proposal 

1 

2 

3 

4 

(44) 

Award  Fee 

1 

2 

3 

4 

Other  Similar  Contractual  Programs, 

(Please  Identify) 

(45)  _  12  3  4 

Would/ does  the  inclusion  in  a  contract  of  any  of  the  contractual  approaches 
listed  in  Question  6  cause  the  Quality  Assurance  activities  in  your  firm  to  staff 
or  plan  actions  in  other  than  the  customary  manner? 


(45.1)  YES  _  WHICH  ONE(S) 


NO 
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10. 


11 


If  any  of  the  following  factors  have  a  degrading  impact  on  the  quality 
of  materiel  you  produce,  please  circle  the  appropriate  number. 


NO  SLIGHT  SIGNIFICANT  SERIOUS 

DEGRADATION  DEGRADATION  DEGRADATION  DEGRADATION 


(46) 

Rush  Job/Schedule 

1 

2 

3  . 

4 

(47) 

Subcontractors 

1 

2 

3 

4 

(48) 

Poor  Working  Conditions 

1 

2 

3 

4 

(49) 

Poor  Specifications 

1 

2 

3 

4 

(50) 

Poor  Morale 

1 

2 

3 

4 

(51) 

Job  Insecurity 

1 

2 

3 

4 

(52) 

Lack  of  Pri de  in  Work 

1 

2 

3 

4 

(53) 

Lack  of  Skill s 

1 

2 

3 

4 

Other,  Please  Identify 
(54) 


Do  you  believe  it  is  realistic  to  include  in  a  contract  a  quality  level  to 
be  achieved? 


(55) 


YES 


NO 


If_the  answer  to  9  is  Yps  ,  what  term  of  specificity  would  you  recommend? 

(56)  Performance  Specification 

(57)  Market  Grade 

(58)  Design  Specification 

(59)  Brand  Name 

(60)  Catalog  Description 

(61 )  Part  Number 

(62)  Acceptable/Unacceptable  Defect  Levels 

(63)  Other,  Please  specify 


Would  you  describe  the  materiel  that  your  firm  builds'as  exceeding  customer 
specified  quality  requi rements? 


(64) 


YES 


NO 
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12.  Do  you  feel  that  the  inclusion  of  life  cycle  CGSt  type  of  incentives 

e.g.,  Reliability  Improvement  Warranty  or  fixed  price  maintenance  options 
in  design/ development  contracts,  would/does  result  in  measurable  improvement 
in  the  quality  of  materiel  normally  produced  by  your  firm? 

(65)  YES  _  NO 


13. 


If  you  were  to  assign  responsibilities  for  product  defects  which 
occur  in  your  plant,  what  proportion  of  that  responsibility  would  you  assign 
to  each  of  the  following  categories  of  personnel? 

(66)  Corporate  or  Top  Level  Management  _  % 

(67)  Middle  Level  Management  _ _ % 

(68)  First  Level  Supervisors  _ _ % 

(69)  Operators  or  Non-Supervisory  _ _ % 

(70)  Design  Engineers  % 


TOTAL 


100  % 


14.  How  do  you  rate  the  following  management  techniques  or  methods  to  assure 

thatyour  firm  is  obtaining  materiel  from  suppliers  that  conforms  to  your 
requi rements.  Please  circle  the  appropriate  numbers. 


(71) 

(72) 

(73) 

(74) 

(75) 

(76) 


NO 

BENEFIT 

SLIGHT 

BENEFIT 

SIGNIFICANT 

BENEFIT 

BEST 

METHOD 

Your  Inspectors  in  Supplier’s  Plant 

1 

2 

3 

4 

Receiving  Inspection 

1 

2 

3 

4 

Supplier  Surveys 

1  ' 

2 

3 

4 

Clear  Definition  of  Requirements 

1 

2 

3 

4 

Financial  Incentives 

1 

2 

3 

4 

Psychological  Incentives 

1 

2 

3 

4 
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15.  How  do  you  rank  the  following  factors  in  terms  of  a  problem  causing  the 

production  of  non-conforming  materiel?  Please  circle  the  appropriate  number. 


* 

NO 

PROBLEM 

SLIGHT 

PROBLEM 

SIGNIFICANT 

PROBLEM 

SERIOUS 

PROBLEM 

(77) 

Poor  Working  Conditions 

1 

2 

3 

4 

(78) 

Low  Pay 

1 

2 

3 

4 

(79) 

Poor  Instructions 

1 

2 

3 

4 

(80) 

Lack  of  Motivation 

1 

2 

3 

4 

(81) 

Lack  of  Interest 

1 

2 

3 

4 

(82) 

Tight  Schedules 

1 

2 

3 

4 

(83) 

Low  Profit 

1 

2 

3 

4 

(84) 

Subcontractors 

1 

2 

3 

4 

(85) 

Poor  Specification  of 
Requi  remen  ts 

1 

2 

3 

4 

16.  What  do  you  believe  is  the  single  most  effective  motivational  device  that 

can  be  incorporated  into  a  contract/subcontract? 

(86)  _ 


17.  Do  you  have  a  specific  method  or  procedure  for  determining  that  a  con¬ 

tractor's/subcontractor's  products  are  such  poor  quality  that  no  more  contracts 
will  be  awarded?  * 

(87)  YES  _  NO _ BASIS  IS  VARIABLE  _ 

If  your  answer  is  Yes,  please  describe  your  method  briefly. 
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If  the  following  factors  are  considered  to  be  measures  of  quality 
direct  or  indirect,  please  rate  them  in  one  of  the  indicated  categorie 
circling  the  appropriate  number. 


NO 

MEASURE 

POOR 

MEASURE 

GOOD 

MEASURE 

(88) 

Engineering  Changes 

1 

2 

3 

(89) 

Defect  Levels  at  Acceptance 
Decision  Points 

V 

2 

3 

(90) 

In-Process  Defect  Rates 

1 

2 

3 

(91) 

Compl iance  wi th.  Delivery 
Schedules 

1 

2 

3 

(92) 

Variations  from  Life  Cycle 
Cost  Projections 

1 

2 

3 

(93) 

Defects  Found  at  Customer 
Delivery  Points 

1 

2 

3 

(94) 

Customer  Complaints  During 
Usage 

1 

2 

3 

(95) 

Process  Control  Effective¬ 
ness 

1 

2 

3 

(96) 

Product  Qualification 

1 

2 

3 

(97) 

Performance  Testing 

1 

2 

3 

(93) 

Receiving  Inspection 

Results 

1 

2 

3 

(99) 

Audit  Findings 

1 

2 

3 

(100) 

Scrap  and  Rework  Costs 

1 

2 

3 

(101) 

Materiel  Review  Board 

Actions 

1 

2 

3 

(102) 

Warranty  Costs 

1 

2 

3 

(103) 

Field  Maintenance  Costs 

1 

2 

3 

Please  list  and  evaluate  any  other  factors 
you  may  consider  to  be  measures  of  quality 

(104)  _ _ _  i  2 


,  either 
s  by 

BEST 

MEASURE 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 

4 


f 


If  your  firm  has  been  provided  Government  furnished  materiel,  i.e., 
component,  subsystem,  etc.,  how  do  you  perceive  its  impact  on  the  quality 
of  the  delivered  end  item,  as  compared  to  end  item  quality  where  you 
provide  all  materiel? 

(105)  Improved  _ _ 

(106)  Degraded  _ 

(107)  No  Difference 

20.  Does  your  company  have  an  employee  motivation  program? 

008)  YES  _  NO  _ 

21*  _  How  do  you  rate  the  following  management  practices  in  terms  of  their 

ability  to  motivate  employees  to  strive  for  high  quality  workmanship  levels7 
Please  circle  the  appropriate  number. 


NO 

BENEFIT 

SLIGHT 
BENEFIT  ' 

SIGNIFICANT 

BENEFIT 

BEST 

METHOD 

(109) 

Increased  Pay 

1 

2 

3 

4 

(no) 

Official  Recognition 

1 

2 

3 

4 

(111) 

Improved  Working  Conditions 

1 

2 

3 

4 

(112) 

Varied  Duties 

1 

2 

3 

4 

(113) 

Suggestion  Programs 

1 

2 

3 

4 

(114) 

Long-Term  Contracts 

1 

2 

3 

4 

(115) 

Special  Bonus  Or  Other 

Cash  Awards 

1 

2 

3 

4 

Other,  Please  Specify 

(116) 

1 

2 

3 

4 

Does 

following 

the  accounting  system 
categories  of  quality 

in  your  firm  accurately  account 
costs? 

for  all  of  the 

(117) 

Defect  Prevention 

YES 

NO 

(118) 

Defect  Detection 

YES 

NO 

(119) 

Defect  Correction 

(Including  Warranty  Costs) 

YES 

NO 
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23.  Please  identify  the  emphasis  government  contract  managers  place  on  the 

following  contract  objectives  by  circling  the  appropriate  number. 


- 

LOW 

EMPHASIS 

MODERATE 

EMPHASIS 

HIGH 

EMPHASIS 

(120) 

Schedule 

1 

2 

3 

021) 

Cost 

1 

2 

3 

(122) 

Quality  of  Materiel 

1 

2 

3 

24.  What  are  your  recommendations  for  improving  the  quality  and  reliability 

of  the  U.S.  Air  Force  Materiel? 

(123) 
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RESPONDENTS  CONFIDENTIAL  INFORMATION  SECTION 


1. 


2. 


3. 


PART  II 

Please  circle  the  number  that  best  describes  your  functional  area  of 
responsibil ity . 

(124)  Contract  Administration 

(125)  Marketing 

(126)  Purchasing 

(127)  Engineering 

(128)  Quality  Assurance 

(129)  Reliability 

(130)  Maintainability 

(131)  Other: _ _ 

Please  circle  the  number  that  best  describes  your  position  in  your  firm 
organization. 

(132)  Corporate  or  Top  Level  Management 

(133)  Middle  Level  Management 

(134)  First  Level  Supervison 

(135)  Non  Supervisory  e.g.,  Engineer,  Assembly,  Inspector 

Please  circle  the  type(s)  of  business  which  best  describes  your 
organization: 


Aerospace  Product-Related  Producer 

(136)  For  Government 

(137)  For  Ai rl ines 

\133)  por  other  Commercial 
Aerospace  Product-Related  User/B uyer 

(139)  For  Government 

(140)  For  Ai rl i nes 

(141)  For  Other  Commercial 


Non-Aerospace  User/Buyer 

(142)  Research  Organization 

(143)  Trade  or  Industry  Association 

(144)  Government  Organization 
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4. 


* 


Please  indicate  what  area(s)  best  describe  your  company's  activities. 

(145)  Research  Hardware  Based 

Non-Hardware  Based 

(146)  Manufacturing  Prime 

Subcontractor  ~ 

(147)  Customer  Sales  Mix  Government  [%) 

Commercial  {%)  ~ 

How  would  you  define  your  firm's  business  and  the  market  environment  in  whirh 
your  firm  operates?  Circle  all  that  aoolv. 

(148)  Limited  Line  of  Specialized  Products 

(149)  Broad  Line  of  Products 

(150)  Few  but  Similar  Competitors 

(151)  Many  and  Diverse  Competitors 

(152)  High  Competition 

(153)  Low  Competition 

(154)  AF  is  a  Major  Customer 

(155)  AF  is  a  Minor  Customer 

(156)  Government  (Defense)  is  Major  Customer 
v!57)  Government  (Non-Defense)  is  Major  Customer 

(158)  Government  (Defense)  is  Minor  Customer 

(159)  Government  (Non-Defense)  is  Minor  Customer 

(160)  Your  Firm  is  a  Major  Supplier  in  its  Market 

(161)  There  is  no  Single  Major  Supplier  in  Your  Firm's  Market 

(162)  Firm  is  Defined  as  a  Small  Business 

(163)  Firms  age  (in  years)  is  0-5;  6-10;  11-15;  16-20;  Greater  Than  20  Years  Old 
(Please  Circle  Approximate  Age) 

(164)  Typical  Contract  Size  Over  Past  Five  Years  is-  Greater  Than 
$10,000;  $100,000;  $500,000;  $1  Million;  $50  Million 
(Please  Circle  Approximate  Size) 

(165)  Market  is  About  an  Even  Mix  of  Defense  and  Non-Defense 
Procurements 
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6. 


May  we  contact  you  at  some  future  date  on  a  confidential  basis  to  further 
discuss  the  subject  matter  of  this  questionnaire? 

(166)  YES  _  NO  _ 


7.  Would  you  like  to  receive  a  copy  of  the  findings  of  this  survey? 

(167)  YES  _ NO _ 

8.  OPTIONAL  -  We  would  appreciate  a  response  here,  but  it  is  not  required. 

Name/Title  :  _ 

Address  :  _ _ _ 

Telephone  Number 

Firm  :  _ 

Operating  Division  :  _ 


We  appreciate  your  assistance  in  answering  this  questionnaire. 


*********** 
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PART  III 


PARTICIPATING  ORGANIZATIONS 


The  following  lists  identifies  the  industrial  and  government 
organizations  that  responded  to  the  questionnaire  or  otherwise 
participated  in  the  data  collection  phase  of  the  investigation. 


A.  INDUSTRIAL  ORGANIZATIONS  (Questionnaire  Respondents) 


t 


Aerojet-General  Corporation 
Aerojet  El ectrosystems  Company 
P.0.  Box  296- 

Azusa,  California  91702 

Aerojet-General  Corporation 
Aerojet  Liquid  Rocket  Company 
P.0.  Box  13222 

Sacramento,  California  95813 

Aerojet-General  Corporation 
Aerojet  Solid  Propulsion  Company 
P.0.  Box  13400 

Sacramento,  California  95813 

Bath  Iron  Works  Corporation 
700  Washington  Street 
Bath,  Maine 

Boeing  Vertol  Company 
P.0.  Box  16868 

Philadelphia,  Pennsylvania  19142 

Caterpillar  Tractor  Company 
Product/Business  Research  Division 
Peoria,  Illinois  61629 


32901 

ESL,  Inc. 

495  Java  Drive 

Sunnyvale,  California  94086 


General  Electric  Company 

Aerospace  Electronics  Systems  Department 

French  Road 

Utica,  New  York  1  3503 

General  Electric 
Space  Division 
P.0.  Box  8555 

Philadelphia,  Pennsylvania  19085 


DBA  Systems,  Inc. 
P.0.  Drawer  550 
Melbourne,  Florida 


The  Garrett  Corporation 
9851-9951  Sepulveda  Boulevard 
Los  Angeles,  California  90009 

General  Dynamics  Corporation 
Fort  Worth  Division 
P.0.  Box  748 

Fort  Worth,  Texas  76101 


Chrysler  Corporation 

Florida  Defense-Space  Operations 

P.0.  Box  7806 

Cape  Canaveral,  Florida  32920 

Clark  Equipment  Company 
General  Offices 
Circle  Drive 

Buchanan,  Michigan  49107 

Collins  Telecommunications 
P.0.  Box  10462 
Dallas,  Texas  75207 

Con  Edison 

4  Irving  Place 

New  York,  New  York  10003 

Consolidated  Diesel  Electric  Co. 
1700  Post  Road 

Old  Greenwich,  Connecticut  06870 

Cummins  Engine  Company,  Inc. 

1000  5th  Street 
Columbus,  Indiana  47201 


General  Electric  Company- 

Armament  Systems 

Burlington,  Connecticut  05401 

Grumman  Corporation 
1111  Steward  Avenue 
Beth page.  New  York  11714 

Harris  Corporation 
Box  37 

Melbourne,  Florida  32901 
HETRA 

P.0.  Box  970 

Melbourne,  Florida  32901 

Hewlett  Packard  Company 

1 501  Page  Mill  Road 

Palo  Alto,  California  94304 

Honeywell  Radiation  Center 
2  Forbes  Road 
Lexington,  Massachusetts 

Honeywell,  Inc. 

Government  &  Aeronautical  Products  Division 
2600  Ridgway  Road 
Minneapolis,  Minnesota  55413 
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Hughes  Aircraft  Company 
Culver  City,  California  90230 

Hughes  Aircraft  Company 
El  Segundo  Manufacturing  Division 
P.0.  Box  92426 

Los  Angeles,  California  90009 

IBM  Corporation 
Bodle  Hill  Road 
Owego,  New  York  13827 

IBM  Corporation 
Federal  Systems  Division 
9500  Godwin  Drive 
Manassas,  Virginia  22110 

International  Harvester  Company 

2200  Pacific  Highway 

San  Diego,  California  92101 

International  Telephone  and  Telegraph 

100  Kingland  Road 

Clifton,  New  Jersey  07014 

Kaiser  Aerospace  &  Electronics 
San  Leandro  Plant 
880  Doolittle  Drive 
San  Leandro,  California  94577 

Walter  Kidd  &  Company,  Inc. 

9  Brighton  Road 

Clifton,  New  Jersey  07015 

Kurz  and  Root  Company,  Inc. 

236  E.  N.  Island  Street 
Appleton  St.,  Wisconsin 

Lear  Siegler,  Incorporated 
Astronics  Division 
3171  South  Bundy  Drive 
Santa  Monica,  California  90406 

Leeds  and  Northrup  Company 
North  Wales,  Pennsylvania  19454 

Litton  Systems,  Inc. 

Guidance  &  Control  Systems  Division 

5500  Canoga  Avenue 

Woodland  Hills,  California  91364 

Lockheed  Aircraft  Corporation 
Lockheed  California  Company 
P.0.  Box  551 

Burbank,  California  91520 


Lockheed  Aircraft  Corporation 
Lockheed  Georgia  Company 
86  South  Cobb  Drive 
Marietta,  Georgia  30063 

Lockheed  Aircraft  Corporation 
Lockheed  Shipbuilding 
2929-1 6th  Avenue 
Seattle,  Washington  98134 

Martin  Marietta  Corporation 
Canaveral  Operations 
Cocoa  Beach,  Florida  32931 

Martin  Mari etta  Corporation 

P.0.  Box  179 

Denver,  Colorado  80201 

Martin  Marietta  Corporation 
P.0.  Box  29304 

New  Orleans,  Louisianna  70189 

Martin  Marietta  Corporation 
Orlando,  Florida 

McDonnell  Douglas  Corporation 
Douglas  Aircraft  Company,  Long  Beach 
3855  Lakewood  Boulevard 
Long  Beach,  California  90801 

Menasco  Manufacturing  Company 

Texas  Division 

P.0.  Box  7565 

Fort  Worth,  Texas  76111 

NCR  Corporation 

1700  S.  Patterson  Boulevard 

Dayton,  Ohio  45479 

Northrup  Corporation 
3901  West  Broadway 
Hawthorne,  California  90250 

Northrup  Corporation 
600  Hicks  Road 

Rolling  Meadows,  Illinois  60008 

Perkin-Elmer  Corporation 
2771  N.  Gary 

Pemona,  California  91761 

Piper  Aircraft  Corporation 

Vero  Beach  Division 

Vero  Beach,  Florida  32960 


Pneumo  Corporation 
2220  Palmer  Avenue 
Kalamazoo,  Michigan  49001 

Q-Bit  Corporation. 

311  Pacific  Avenue 
Palm  Bay,  Florida  32905 

Raytheon  Company 

141  Spring  Street 

Lexington,  Massachusetts  02173 

RCA  Corporation 
Aerospace  Systems  Division 
P.0.  Box  588 

Burlington,  Massachusetts  01801 

RCA  Corporation 
EASD 

8500  Balboa  Boulevard 
Van  Nuys ,  California  91406 

RCA-  Corporation 
Government  Systems 
Route  38 

Cherry  Hill,  New  Jersey  08358 
RCA  Corporation 

Missile  &  Surface  Radar  Division 
Moorestown,  New  Jersey  08057 

Rockwell  Collins  Avionics 
400  Coll  ins  Road,  N.E. 

Cedar  Rapids,  Iowa  52406 

Sanders  Associates,  Inc. 

Federal  Systems  Group 

95  Canal  Street 

Nashua,  New  Hampshire  03061 

Santa  Barbara  Research  Center  (Hughes) 

75  Coromar  Drive 

Goleta,  California  93017 

GTE  Syl vania,  Inc. 

189  "B"  Street 

Needham,  Massachusetts  02194 


TRAC0R,  Inc. 

6500  Tracor  Lane 
Austin,  Texas  78721 

Vought  Aeronautics  Company 
1525  Elm 

Dallas,  Texas  75201 

Western  Electric  Company 
Box  20046 

Greensboro,  North  Carolina  27420 

Westinghouse  Electric  Corporation 
P.0.  Box  1521 

Baltimore,  Maryland  21203 

Westinghouse  Electric  Corporation 
Power  Systems  Product  Division 
875  Greentree  Road 
Pittsburg,  Pennsylvania 

Wilcox  Electric  Inc. 

1400  Chestnut  Street 
Kansas  City,  Missouri  64127 
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INDUSTRIAL  ORGANIZATIONS  VISITED 


Aerojet-General  Corporation 
Sacramento,  California 


Communications  Satellite  Corporation 
Gaithersburg,  Maryland 


General  Dynamics 
Fort  Worth,  Texas 


Harris  Corporation 
Melbourne,  Florida 


Hughes  Aircraft 
El  Segundo,  California 


Lockheed  Missile  and  Space  Company 
Sunnyvale,  California 


Northrup  Corporation 
Newberry  Park,  California 


C.  GOVERNMENT  OFFICES  VISITED  OR  CONTACTED 


Deputy  Under  Secretary  of  Defense  (R&E)  (Acquisition  Policy)  Office 

Director,  Contract  Acquisition 
Director,  Material  Acquisition  Policy 

Hq.  USAF,  Directorate  of  Contracting  and  Acquisition  Policy 

Hq.  Air  Force  Systems  Command,  Directorate  of  Quality  Assurance  Policy 

Hq.  Air  Force  Logistics  Command 

Office  of  the  Assistant  to  the  Commander  for 
Quality  Assurance 

DCS/Logistics  Operations,  Directorate  of  Engineering 
Services 

Aeronautical  Systems  Division 

Office  of  the  Deputy  for  Contracting  and  Manufacturing 
Office  of  the  Deputy  for  Aeronautical  Equipment 
Office  of  the  Deputy  for  Engineering 
Office  of  the  Deputy  for  the  F-16 
Office  of  the  Deputy  for  the  F-15 

Electronics  Systems  Division 

Product  Assurance  Directorate 
Directorate  of  Engineering 
Directorate  of  Manufacturing 

Space  and  Missile  System  Organization 

Quality  Assurance  Directorate 
Reliability  and  Parts  Control 
Advanced  Development 
System  Effectiveness 
Directorate  GPS  Contracts 

Hq.  Defense  Logistics  Agency 

Executive  Directorate  for  Quality  Assurance 
Defense  Material  Specifications  and  Standards  Office 
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Air  Force  Contract  Management  Division 

Quality  Assurance  Directorate 
Contract  Administration 
Engineering  Program  Support 
Manufacturing 

Armament  Development  and  Test  Center 

Quality  Assurance  Office 
Reliability  Office 

Department  of  the  Navy 

Special  DCNM  for  Reliability,  Maintainability  and  Quality 
Assurance 

Department  of  Energy 

Procurement  Support  Office 
Federal  Aviation  Administration 
Industrial  Division 
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APPENDIX  B 
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APPENDIX  C 


OFFICIAL  DOCUMENTS 


This  Appendix  contains  a  listing  of  the  official  documents 
which  impact  on  the  area  of  study.  While  there  are  numerous 

other  non-quality  assurance  Directives,  Specifications, 
Standards,  etc.  which  affect  quality  and  reliability,  those 
that  appear  here  are  selected  as  being  the  most  pertinent 
and  significant. 


OFFICIAL  DOCUMENTS 


<*• 
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Office  of  Federal  Procurement  Policy,  Phamphlet  No .  1 ,  August  1976 
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Department  of  Defense  Directive  5000.2,  March  19,  1980 
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Non-Conforming  Material,  March  21,  1975 
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February  1  ,  1 974 

Draft  AFSC  Quality  Assurance  Guidebook,  February  1,  1980 
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Air  Force  Contract  Management  Division  Regulation  178-1,  Contractor  Management 
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